Skip to content

LogSentinel Collector Overview

The LogSentinel Collector is а component that gets installed on-premise to listen to a configured set of log sources. It can be installed on Linux and Windows and supports the following types of sources:

LogSentinel allows collecting log records from industry-leading vulnerability scanners.

  • Log files an arbitrary text file can be collected and sent, line by line, to the LogSentinel SIEM service. These typically application logs and logs by systems like application servers, accounting software, CRMs, ERPs (e.g. SAP), big data processing, or any other non-standard software. Formats like CSV, XML, JSON as well as non-standard ones are supported
  • Database tables - if you store audit trail inside relational database tables, you can configure queries that periodically fetch new entries and send them to the LogSentinel SIEM service
  • Syslog - collect syslog messages sent by any appliances pointed to the collector, in any format and variation (RFC 3164, RFC 5424; CEF, LEEF; UDP, TCP/TLS, RELP)
  • NetFlow/IPFIX/sFlow - act as a NetFlow/IPFIX/sFlow collector and forward the flows to the SIEM
  • SSH - connect to any system over SSH and listen to text files or the activities of the logged in users
  • Database logs files - if database query logs are enabled, the collector listens to newly issued queries and sends them to the LogSentinel SIEM service
  • MS SQL audit trail – if MS SQL audit trail is enabled, the collector can be configured to listen to it and forward the events
  • MS SQL change tracking – if MS SQL change tracking is enabled, the collector can be configured to listen to it and forward the change events
  • Oracle audit trail – if Oracle audit trail is enabled, the collector can be configured to listen to it and forward the events
  • Access logs – the standard web server access log files can be parsed (in Common Log Format and Extended Log Format) and sent to LogSentinel SIEM
  • Linux audit log – the native Linux audit log file can be tailed and forwarded to LogSentinel SIEM
  • Windows event logs – the Windows event logs (including all categories – Application, Security, System) can be read continuously and sent to LogSentinel SIEM
  • Directory changes - any changes in a directory (new files, removal of files, modification of files) can be tracked using this collector configuration
  • PostgreSQL - the collector can collect the audit logs generated by pgaudit
  • MySQL - the collector can collect the audit logs generated by the audit log plugin
  • Teradata - if auditing is enabled on Teradata, the collector can query the logs and forward them
  • Hadoop - the collector can parse and forward Hadoop security logs
  • AxonDB logs – AxonDB is a special type of non-relational database. We support its custom log format
  • VSphere logs – the VCenter events and the system events logs (if HostHealthStatusSystem is enabled) can be tracked using this collector configuration
  • SNMP traps - the collector can act as an SNMP manager and receive SNMP traps
  • Network capture - the collector can do packet capturing if it receives traffic sent via a SPAN port / morriring and transform them to flow data
  • Vulnerability scans - the collector can perform vulnerability scans across the detected assets and report vulnerability results as events to the SIEM. Optionally, metasploit resource scripts can be automatically executed, if configured

Any combination of the above can be configured. Note that for most target types the collector can be installed on a different machine than the actual log source, thus supporting full agentless collection. For example, in case of database tables or database audit trail, the collector can be installed on another machine that connects to the database server via a database connection string and credentials.

Full configuration details can be seen here.

All communication between the collector and the LogSentinel SIEM is encrypted.

Configuration and installation are done through scripts provided by us and you can follow the steps here.

In order to find existing logs and tables to monitor and send to LogSentinel SIEM, you can use our simple open source scan-logs script tool.

Collecting remote files

Text files residing on servers other than the one where the LogSentinel Collector is installed is typically a challenge for log collection. Please refer to our Collecting Remote Files section for more details.

Collector queues

In case of lack of connectivity to the SIEM, the connector can keep a queue of the data or keep trying to read the same data which it failed sending, for an extensive period of time. After connection is restored, the unsent logs are sent in batches to the SIEM.

Queues are kept in memory (for syslog and netflow sources) until a certain limit is reached and then are stored in disk. While collectors usually don't require much disk space, such scenarios should be taken into account when provisioning collector disk space.

Custom connectors

Custom connectors can be easily added to LogSentinel Collector. This can be done in two ways:

  • Python scripts to extend a basic Python executor, useful to parse non-standard formats for text files, syslog messages, etc.
  • Java extensions - our collector is open-source and can be extended (by us or by a partner) to accommodate any use-case.

Collector Placement and Sizing

When considering where to place your Collectors, keep in mind that your bandwidth and network architecture will influence the number of Collectors that you need in your organization and where you should place them. Generally, you should deploy the Collectors close to the logs that will be pulled or sent and close to the endpoints that they will be scanning.

LogSentinel recommends a maximum of 80 event sources for each Collector, depending on the following:

  • The size of the event sources being added
  • The amount of CPU memory available to the Collector
  • The amount of VM resources available to the Collector
  • The amount of disk space available to the Collector


Tip: Keep up to 50-60 event sources per Collector and distribute event sources over multiple Collectors

The capacity of a collector depends on multiple factors. While the maximum recommended is 80 event sources for each Collector, it can be more convienent to keep up to 50-60 event sources per collector to prevent data collection issues.

Distributing event sources over multiple collectors is always a good practice.

Collector Location Size

Number of Endpoints/Agents

Number of Event Sources on the Collector

Recommended Minimum CPU

Recommended Minimum RAM

Recommended Minimum Disk Space


Up to 500

1 - 10


8 GB

60 GB


Up to 2,400

10 - 50


8 GB

80 GB


Up to 600 per CPU core

50 - 80*


16 GB

100 GB

*If you have more than 80 event sources, you should split your event sources across multiple collectors.