Edit File: rsyslog_conf_modules.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html><head><title>Modules - rsyslog.conf</title></head> <body> <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> <h1>Modules</h1> <p>Rsyslog has a modular design. This enables functionality to be dynamically loaded from modules, which may also be written by any third party. Rsyslog itself offers all non-core functionality as modules. Consequently, there is a growing number of modules. Here is the entry point to their documentation and what they do (list is currently not complete)</p> <p>Please note that each module provides configuration directives, which are NOT necessarily being listed below. Also remember, that a modules configuration directive (and functionality) is only available if it has been loaded (using $ModLoad).</p> <p>It is relatively easy to write a rsyslog module. <b>If none of the provided modules solve your need, you may consider writing one or have one written for you by <a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a> </b>(this often is a very cost-effective and efficient way of getting what you need). <p>There exist different classes of loadable modules: <ul> <li><a href="rsyslog_conf_modules.html#im">Input Modules</a> <li><a href="rsyslog_conf_modules.html#om">Output Modules</a> <li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a> <li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a> <li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a> <li><a href="rsyslog_conf_modules.html#lm">Library Modules</a> </ul> <a name"im"></a><h2>Input Modules</h2> <p>Input modules are used to gather messages from various sources. They interface to message generators. <ul> <li><a href="imfile.html">imfile</a> - input module for text files</li> <li><a href="imrelp.html">imrelp</a> - RELP input module</li> <li>imudp - udp syslog message input</li> <li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li> <li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li> <li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li> <li>immark - support for mark messages</li> <li><a href="imklog.html">imklog</a> - kernel logging</li> <li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li> <li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li> <li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li> <li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li> </ul> <a name"om"></a><h2>Output Modules</h2> <p>Output modules process messages. With them, message formats can be transformed and messages be transmitted to various different targets. <ul> <li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> <li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> <li><a href="omrelp.html">omrelp</a> - RELP output module</li> <li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li> <li>omgssapi - output module for GSS-enabled syslog</li> <li><a href="ommysql.html">ommysql</a> - output module for MySQL</li> <li>ompgsql - output module for PostgreSQL</li> <li><a href="omlibdbi.html">omlibdbi</a> - generic database output module (Firebird/Interbase, MS SQL, Sybase, SQLLite, Ingres, Oracle, mSQL)</li> <li><a href="ommail.html">ommail</a> - permits rsyslog to alert folks by mail if something important happens</li> <li><a href="omprog.html">omprog</a> - permits sending messages to a program for custom processing</li> <li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li> <li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li> <li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li> <li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li> </ul> <a name="pm"></a><h2>Parser Modules</h2> <p>Parser modules are used to parse message content, once the message has been received. They can be used to process custom message formats or invalidly formatted messages. For details, please see the <a href="messageparser.html">rsyslog message parser documentation</a>. <p>The current modules are currently provided as part of rsyslog: <ul> <li>pmrfc5424[builtin] - rsyslog.rfc5424 - parses RFC5424-formatted messages (the new syslog standard) <li>pmrfc3164[builtin] - rsyslog.rfc3164 - the traditional/legacy syslog parser <li>pmrfc3164sd - rsyslog.rfc3164sd - a contributed module supporting RFC5424 structured data inside RFC3164 messages (not supported by the rsyslog team) <li><a href="pmlastmsg.html">pmlastmsg</a> - rsyslog.lastmsg - a parser module that handles the typically malformed "last messages repated n times" messages emitted by some syslogds. </ul> <a name="mm"></a><h2>Message Modification Modules</h2> <p>Message modification modules are used to change the content of messages being processed. They can be implemented using either the output module or the parser module interface. From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special. <p>Currently, there do not exist any such modules, but could be written with the methods the engine provides. They could be used, for example, to: <ul> <li>anonymize message content <li>add dynamically computed content to message (fields) </ul> <p>Message modification modules are usually written for one specific task and thus usually are not generic enough to be reused. However, existing module's code is probably an excellent starting base for writing a new module. Currently, the following modules existin inside the source tree <ul> <li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside the tag to correct the original sender system and priority of messages. Implemented via the output module interface. </ul> <a name="lm"></a><h2>String Generator Modules</h2> <p>String generator modules are used, as the name implies, to generate strings based on the message content. They are currently tightly coupled with the template system. Their primary use is to speed up template processing by providing a native C interface to template generation. These modules exist since 5.5.6. To get an idea of the potential speedup, the default file format, when generated by a string generator, provides a roughly 5% speedup. For more complex strings, especially those that include multiple regular expressions, the speedup may be considerably higher. <p>String generator modules are written to a quite simple interface. However, a word of caution is due: they access the rsyslog message object via a low-level interface. That interface is not guaranteed yet to stay stable. So it may be necessary to modify string generator modules if the interface changes. Obviously, we will not do that without good reason, but it may happen. <p>Rsyslog comes with a set of core, build-in string generators, which are used to provide those default templates that we consider to be time-critical: <ul> <li>smfile - the default rsyslog file format <li>smfwd - the default rsyslog (network) forwarding format <li>smtradfile - the traditional syslog file format <li>smfwd - the traditional syslog (network) forwarding format </ul> <p>Note that when you replace these defaults be some custom strings, you will loose some performance (around 5%). For typical systems, this is not really relevant. But for a high-performance systems, it may be very relevant. To solve that issue, create a new string generator module for your custom format, starting out from one of the default generators provided. If you can not do this yourself, you may want to contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development of string generators at a very low price. <p>Note that string generator modules can be dynamically loaded. However, the default ones provided are so important that they are build right into the executable. But this does not need to be done that way (and it is straightforward to do it dynamic). <a name="lm"></a><h2>Library Modules</h2> <p>Library modules provide dynamically loadable functionality for parts of rsyslog, most often for other loadable modules. They can not be user-configured and are loaded automatically by some components. They are just mentioned so that error messages that point to library moduls can be understood. No module list is provided. <h2>Where are the modules integrated into the Message Flow?</h2> <p>Depending on their module type, modules may access and/or modify messages at various stages during rsyslog's processing. Note that only the "core type" (e.g. input, output) but not any type derived from it (message modification module) specifies when a module is called. <p>The simplified workflow is as follows: <p align="center"> <img src="module_workflow.png" alt"rsyslog: loadable modules and message flow"> <p>As can be seen, messages are received by input modules, then passed to one or many parser modules, which generate the in-memory representation of the message and may also modify the message itself. The, the internal representation is passed to output modules, which may output a message and (with the interfaces newly introduced in v5) may also modify messageo object content. <p>String generator modules are not included inside this picture, because they are not a required part of the workflow. If used, they operate "in front of" the output modules, because they are called during template generation. <p>Note that the actual flow is much more complex and depends a lot on queue and filter settings. This graphic above is a high-level message flow diagram. <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body> </html>
Back to File Manager