Collect logs from ModSecurity with Elastic Agent
What is an Elastic integration?
This integration is powered by Elastic Agent. Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Refer to our documentation for a detailed comparison between Beats and Elastic Agent.
Prefer to use Beats for this use case? See Filebeat modules for logs or Metricbeat modules for metrics.
See the integrations quick start guides to get started:
This integration periodically fetches audit logs from Modsecurity servers. It can parse audit logs created by the HTTP server.
The logs were tested with ModSecurity v3 with nginx connector and ModSecurity v3 with Apache Connector. Change the default ModSecurity logging format to json as per configuration.
SecAuditLogParts ABDEFHIJZ
SecAuditLogType Serial
SecAuditLog /var/log/modsec_audit.json
SecAuditLogFormat JSONBe careful to drop the list of all rules that matched for the transaction (K) in SecAuditLogParts. That part can make raw logs too long to parse.
The Audit Log dataset collects Modsecurity Audit logs.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.domain | The domain name of the destination system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location.lat | Longitude and latitude. | geo_point |
destination.geo.location.lon | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.dataset | Event dataset | constant_keyword |
event.module | Event module | constant_keyword |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
http.request.body.content | The full HTTP request body. | wildcard |
http.request.body.content.text | Multi-field of http.request.body.content. | match_only_text |
http.request.bytes | Total size in bytes of the request (body and headers). | long |
http.request.method | HTTP request method. The value should retain its casing from the original event. For example, GET, get, and GeT are all considered valid values for this field. | keyword |
http.request.mime_type | Mime type of the body of the request. This value must only be populated based on the content of the request body, not on the Content-Type header. Comparing the mime type of a request with the request's Content-Type header can be helpful in detecting threats or misconfigured clients. | keyword |
http.request.referrer | Referrer for this HTTP request. | keyword |
http.response.body.content | The full HTTP response body. | wildcard |
http.response.body.content.text | Multi-field of http.response.body.content. | match_only_text |
http.response.bytes | Total size in bytes of the response (body and headers). | long |
http.response.mime_type | Mime type of the body of the response. This value must only be populated based on the content of the response body, not on the Content-Type header. Comparing the mime type of a response with the response's Content-Type header can be helpful in detecting misconfigured servers. | keyword |
http.response.status_code | HTTP response status code. | long |
http.version | HTTP version. | keyword |
input.type | Input type | keyword |
log.file.path | Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. If the event wasn't read from a log file, do not populate this field. | keyword |
log.offset | Log offset | long |
message | human-readable summary of the event | text |
modsec.audit.details | Modsecurity audit details. | flattened |
modsec.audit.messages | Modsecurity audit message. | keyword |
modsec.audit.server | Modsecurity server name. | keyword |
related.ip | All of the IPs seen on your event. | ip |
rule.id | A rule ID that is unique within the scope of an agent, observer, or other entity using the rule for detection of this event. | keyword |
source.address | Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the .address field. Then it should be duplicated to .ip or .domain, depending on which one it is. | keyword |
source.as.number | Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet. | long |
source.as.organization.name | Organization name. | keyword |
source.as.organization.name.text | Multi-field of source.as.organization.name. | match_only_text |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
tags | List of keywords used to tag each event. | keyword |
transaction.id | Unique identifier of the transaction within the scope of its trace. A transaction is the highest level of work measured within a service, such as a request to a server. | keyword |
url.domain | Domain of the url, such as "www.elastic.co". In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the domain field. If the URL contains a literal IPv6 address enclosed by [ and ] (IETF RFC 2732), the [ and ] characters should also be captured in the domain field. | keyword |
url.extension | The field contains the file extension from the original request url, excluding the leading dot. The file extension is only set if it exists, as not every url has a file extension. The leading period must not be included. For example, the value must be "png", not ".png". Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz"). | keyword |
url.fragment | Portion of the url after the #, such as "top". The # is not part of the fragment. | keyword |
url.original | Unmodified original url as seen in the event source. Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path. This field is meant to represent the URL as it was observed, complete or not. | wildcard |
url.original.text | Multi-field of url.original. | match_only_text |
url.path | Path of the request, such as "/search". | wildcard |
url.port | Port of the request, such as 443. | long |
url.query | The query field describes the query string of the request, such as "q=elasticsearch". The ? is excluded from the query string. If a URL contains no ?, there is no query field. If there is a ? but no query, the query field exists with an empty string. The exists query can be used to differentiate between the two cases. | keyword |
url.scheme | Scheme of the request, such as "https". Note: The : is not part of the scheme. | keyword |
user.name | Short name or login of the user. | keyword |
user.name.text | Multi-field of user.name. | match_only_text |
user_agent.device.name | Name of the device. | keyword |
user_agent.name | Name of the user agent. | keyword |
user_agent.original | Unparsed user_agent string. | keyword |
user_agent.original.text | Multi-field of user_agent.original. | match_only_text |
user_agent.os.full | Operating system name, including the version or code name. | keyword |
user_agent.os.full.text | Multi-field of user_agent.os.full. | match_only_text |
user_agent.os.name | Operating system name, without the version. | keyword |
user_agent.os.name.text | Multi-field of user_agent.os.name. | match_only_text |
user_agent.os.version | Operating system version as a raw string. | keyword |
user_agent.version | Version of the user agent. | keyword |
| Version | Details |
|---|---|
1.7.0 | Enhancement View pull request Update package to ECS 8.7.0. |
1.6.1 | Enhancement View pull request Added categories and/or subcategories. |
1.6.0 | Bug fix View pull request Handle sub-second resolution timestamps. Enhancement View pull request Improve error handling. |
1.5.1 | Bug fix View pull request Add support for time zone configuration. |
1.5.0 | Enhancement View pull request Update package to ECS 8.6.0. |
1.4.1 | Bug fix View pull request Remove duplicate fields. |
1.4.0 | Enhancement View pull request Update package to ECS 8.5.0. |
1.3.0 | Enhancement View pull request Adding better extraction of http request headers |
1.2.1 | Enhancement View pull request Use ECS geo.location definition. |
1.2.0 | Enhancement View pull request Update package to ECS 8.4.0 |
1.1.3 | Bug fix View pull request Fix typo in the build/docs/README.md |
1.1.2 | Bug fix View pull request Fix date format, Json issues and apache modesecurity issues |
1.1.1 | Enhancement View pull request Update package name and description to align with standard wording |
1.1.0 | Enhancement View pull request Update package to ECS 8.3.0. |
1.0.0 | Enhancement View pull request Make GA |
0.1.5 | Enhancement View pull request Add documentation for multi-fields |
0.1.4 | Enhancement View pull request Change ownership to correct owner and update versions to support 8.x |
0.1.3 | Bug fix View pull request Regenerate test files using the new GeoIP database |
0.1.2 | Bug fix View pull request Change test public IPs to the supported subset |
0.1.1 | Bug fix View pull request Fix logic that checks for the 'forwarded' tag |
0.1.0 | Enhancement View pull request Initial draft of the package |