FluentD

 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  FluentD : Parse +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  Some of the Fluentd plugins support the <parse> section to specify how to parse the raw data. Parse Section Overview : The parse section can be under <source> , <match> or <filter> section. It is enabled for the plugins that support parser plugin features. <source>   @type tail   # ...   <parse>     # ...   </parse> </source> Parse Plugin Type : The @type parameter of <parse> section specifies the type of the parser plugin. Fluentd core bundles some useful  <parse>   @ type apache2 </parse> Third-party plugins may also be installed and configured.                      

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++   +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ You can also access to a certain portion of a tag using the following notations: tag_parts[N] refers to the Nth part of the tag. tag_prefix[N] refers to the [0..N] part of the tag. tag_suffix[N] refers to the [N..] part of the tag. All indices are zero-based. For example, if you have an incoming event tagged debug.my.app, then tag_parts[1] will represent my. Also in this case, tag_prefix[N] and tag_suffix[N] will work as follows: tag_prefix[0] = debug          tag_suffix[0] = debug.my.app tag_prefix[1] = debug.my       tag_suffix[1] = my.app tag_prefix[2] = debug.my.app   tag_suffix[2] = app

 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Input Plugins   Fluentd has nine (9) types of plugins: https://docs.fluentd.org/input  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++     The in_tail Input plugin allows Fluentd to read events from the tail of text files. Its behavior is similar to the tail -F command. Example Configuration : <source>   @type tail   path /var/log/httpd-access.log   pos_file /var/log/td-agent/httpd-access.log.pos   tag apache.access   <parse>     @type apache2   </parse> </source> Parse : Meaning : t o divide (a sentence) into grammatical parts and identify the parts and their relations to each other . When Fluentd is first configured with in_tail , it will start reading from the tail of that log, not the beginning. Once the log is rotated, Fluentd starts reading the new file from the beginning. It keeps t...

 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++     +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  Interlude Routing : The source submits events to the Fluentd routing engine. An event consists of three entities: tag, time and record. The tag is a string separated by dots (e.g. myapp.access), and is used as the directions for Fluentd internal routing engine. The time field is specified by input plugins, and it must be in the Unix time format. The record is a JSON object

 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  This was the first -- Fluentd configuration with Folder creates  dd/mm/yyyy    messages   secure   kern  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++   <source>   @type syslog   @id input_syslog   port 4889   bind 0.0.0.0   <transport tcp>   </transport>   <parse>     @type syslog     with_priority false     message_format auto   </parse>   emit_unmatched_lines true   severity_key message_severity   source_address_key source_address   tag oci.0.8x8.vo.dev.os.syslog.* </source> <source>   @type tail #  @id input_tail_audit   path /var/log/audit/audit.log   pos_file /var/lib/fluent_oci_outplugin/pos/audit.pos   pos_file_compaction_interval 24h   enable_watch_tim...