What is graylog
Graylog is an opensource log management tool which helps you to collect, index and analyze any machine logs centrally.
Required Components:
i) MongoDB - Database to store the configurations and meta information's.
ii) Elasticsearch - It stores the log messages received from graylog and provide the facility to search them whenever we required.
iii) Graylog server - This does the parsing of logs that are coming from various inputs and provides build-in web interface to handle those logs.
Step 1
Elasticsearch requires java so we need to install java 1st best version is java .
Download the RPM package from online and install it and make sure to export the JAVA_HOME.
Step 2
Install Elasticsearch
First import the GPG key before installation.
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
now create the repo file for the elasticsearch
vi /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=https://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
Now install the package
yum install elasticsearch -y
Elasticsearch Configuration:
need some changes to work with graylog, edit the elasticsearch.yml file
vi /etc/elasticsearch/elasticsearch.yml
change cluster.name
cluster.name: graylog
then add below lines in configuration
script.inline: false
script.indexes: false
script.file: false
change network.host to your ip
network.host: 192.168.1.1
then save & exit
Now start the service of elasticsearch
service elasticsearch start
chkconfig elasticsearch on
service elasticsearch status
Now check the elasticsearch is working or not with following command it should give the cluster name as graylog
curl -X GET http://192.168.1.1:9200
now perform health check if it's get green then your elasticsearch working good.
curl -XGET 'http://192.168.1.1:9200/_cluster/health?pretty=true'
Now elasticsearch heap size default to 256M min 1G max. It's recommended to set ES_HEAP_SIZE to 50% of available RAM size and no more than 31G.
Now open /etc/sysconfig/elasticsearch
change the below lines.
ES_HEAP_SIZE=4G
MAX_OPEN_FILES=640000
Elasticsearch has to keep a lot of files open simultaneously and requires a higher open file limit that the usual operating system defaults allow.
do the following step to increase the limit
echo "elasticsearch soft nofile 64000" >> /etc/security/limits.conf ;/
echo "elasticsearch hard nofile 64000" >> /etc/security/limits.conf
now restart the elasticsearch
service elasticsearch restart
Step 3
Install mongoDB
Setup mongodb repo
vi /etc/yum.repos.d/mongodb.repo
add the below lines in it.
[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.2.asc
Now install the mongodb
yum install mongodb-org -y
now start the service
service mongod start
chkconfig mongod on
Step 4
Install Graylog:
Download and install graylog repository
rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.0-repository_latest.rpm
and change the baseurl from the repo
vi /etc/yum.repos.d/graylog.repo
and replace the baseurl with below mentioned url
[graylog]
name=graylog
baseurl=https://packages.graylog2.org/repo/el/stable/2.2/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-graylog
now install graylog-server
yum install graylog-server -y
now use the following command to create password secret
but before make sure it pwgen was installed or else install epel update and run the below command
yum install pwgen -y
now run the command to generate secret key
pwgen -N 1 -s 96
it will give output like below
OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
and also run the below command to generate root_password_sha2
echo -n yourpassword | sha256sum
it will give output as
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
now open the /etc/graylog/server/server.conf
vi /etc/graylog/server/server.conf
add the password secret
password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
and add root_password_sha2
root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
and remove # from add required details are followed
root_username = admin
root_email = "your mail address"
root_timezone = UTC ===> by default UTC you can change
elasticsearch_discovery_zen_ping_unicast_hosts = 192.168.1.1:9300
is_master = true
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
elasticsearch_shards = 1
elasticsearch_replicas = 0
below for garylog-web interface
rest_listen_uri = http://192.168.1.1:12900
web_listen_uri = http://192.168.1.1:9000/
then save & exit
now change GRAYLOG_SERVER_JAVA_OPTS in /etc/sysconfig/graylog-server min and max size on that
-Xms2G -Xmx4G
save and exit.
now start the graylog-server
service graylog-server start
chkconfig graylog-server on
make sure your selinux and firewall is stopped or else allow the port 9000, 12900 and receiving port like 514 or something.
now open the url in browser http://192.168.1.1:9000
username: admin
password: yourpassword
that's all now you can add hosts and receive messages.