ELK Packetbeat 协议和进程配置项(16th)

Packetbeat 协议部分包含支持每个协议的配置项，如ports, send_request, send_response 和 特定的协议选项。

目前Packetbeat支持以下协议：文章源自运维生存时间-https://www.ttlsa.com/elk/elk-packetbeat-protocols-and-processes-configure-options/

• DNS
• HTTP
• Mysql
• PostgreSQL
• Redis
• Thrift-RPC
• MongoDB
• Memcache

配置如下：文章源自运维生存时间-https://www.ttlsa.com/elk/elk-packetbeat-protocols-and-processes-configure-options/

protocols:
  dns:
    ports: [53]

  http:
    ports: [80, 8080, 8000, 5000, 8002]

  memcache:
    ports: [11211]

  mysql:
    ports: [3306]

  redis:
    ports: [6379]

  pgsql:
    ports: [5432]

  thrift:
    ports: [9090]

通用协议选项

下面的选项可用于所有协议：文章源自运维生存时间-https://www.ttlsa.com/elk/elk-packetbeat-protocols-and-processes-configure-options/

protocols:
  dns:
    ports: [53]

    # include_authorities controls whether or not the dns.authorities field
    # (authority resource records) is added to messages.
    # Default: false
    include_authorities: true
    # include_additionals controls whether or not the dns.additionals field
    # (additional resource records) is added to messages.
    # Default: false
    include_additionals: true

配置实例如下：用的最多是这个，注意看文章源自运维生存时间-https://www.ttlsa.com/elk/elk-packetbeat-protocols-and-processes-configure-options/

protocols:
  http:

    # Configure the ports where to listen for HTTP traffic. You can disable
    # the http protocol by commenting the list of ports.
    ports: [80, 8080, 8000, 5000, 8002]

    # Uncomment the following to hide certain parameters in URL or forms attached
    # to HTTP requests. The names of the parameters are case insensitive.
    # The value of the parameters will be replaced with the 'xxxxx' string.
    # This is generally useful for avoiding storing user passwords or other
    # sensitive information.
    hide_keywords: ["pass", "password", "passwd"]

    # Uncomment the following to export a list of extra HTTP headers. By
    default is none sent.
    send_headers: ["User-Agent", "Cookie", "Set-Cookie"]

    # Uncomment the following to export Cookie or Set-Cookie headers. By
    # default is false.
    split_coookie: true

    # Configure the HTTP header that contains the real IP address.
    real_ip_header: "X-Forwarded-For"

查询参数列表，Packetbeat将自动审查已存的事务。与这些参数相关的值将被替代成 'xxxxx'。默认情况下，是不会更改HTTP信息的。

Packetbeat 有这个选项，是因为不同于 SQL 流量只含有密码的hash值，HTTP 流量可能包含敏感的数据。为了降低安全风险，可配置此选项避免HTTP POST发送特定的参数。

如：

protocols:
  http:
    ports: [80, 8080]
    send_response: true
    include_body_for: ["text/html"]

如果Cookie 或者 Set-Cookie 头部被发送, 该选项控制是否把它们分割成独自值。默认false。例如，设置了该选项， HTTP 响应将成下面的json格式结果：

"response": {
  "code": 200,
  "headers": {
    "connection": "close",
    "content-language": "en",
    "content-type": "text/html; charset=utf-8",
    "date": "Fri, 21 Nov 2014 17:07:34 GMT",
    "server": "gunicorn/19.1.1",
    "set-cookie": { 
      "csrftoken": "S9ZuJF8mvIMT5CL4T1Xqn32wkA6ZSeyf",
      "expires": "Fri, 20-Nov-2015 17:07:34 GMT",
      "max-age": "31449600",
      "path": "/"
    },
    "vary": "Cookie, Accept-Language"
  },
  "phrase": "OK"
}

提取真实IP地址。该信息用于 real_ip 和 client_location 索引字段。

实例如下：

  memcache:
    ports: [11211]
    parseunknown: false
    maxvalues: 0
    maxbytespervalue: 100
    transaction_timeout: 200
    udptransactiontimeout: 200

当启用该选项，迫使memcache文本协议分析器接受未知的命令。

  thrift:
    transport_type: socket
    protocol_type: binary
    idl_files: ["tutorial.thrift", "shared.thrift"]
    string_max_size: 200
    collection_max_size: 20
    capture_reply: true
    obfuscate_strings: true
    drop_after_n_struct_fields: 100

  mongodb:
    send_request: true
    send_response: true
    max_docs: 0
    max_doc_length: 0

procs:
  enabled: true
  monitored:
    - process: mysqld
      cmdline_grep: mysqld

    - process: pgsql
      cmdline_grep: postgres

    - process: nginx
      cmdline_grep: nginx

    - process: app
      cmdline_grep: gunicorn