ELK enterprise application - elk quick build - logstash
1, install JDK
elasticsearch, the operation of logstash depends on the java environment.
Download and unzip the jdk binary package.
tar xf jdk-8u144-linux-x64.tar.gz -C /usr/local
mv /usr/local/jdk1.8.0_144 /usr/local/java
Configure the java environment variable.
Add the following at the end of the ~/.bashrc file:
export JAVA_HOME=/usr/local/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/bin/tools.jar:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
Make the configuration take effect.
source ~/.bashrc
2, install Logstash
It is recommended that the Linux class server download the rmp package installation.
2.1. Download the logstash installation package
touch /etc/default/logstash
ln -s /usr/local/java/bin/java /usr/bin/java
rpm -ivh logstash-6.2.4.rpm
2.2. Configure systemd to start
When installing rpm, the configuration file for creating the startup script is /etc/logstash/startup.options
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Note: When the script fails to start, you can create your own startup script.
[root@l ~]# cat /etc/systemd/system/logstash.service
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
ExecStop=/bin/kill -s QUIT $MAINPID
ExecReload=/bin/kill -s HUP $MAINPID
WorkingDirectory=/usr/share/logstash/bin
WantedBy=multi-user.target
[root@l ~]# systemctl daemon-reload #####Update
[root@l ~]# systemctl list-unit-files |grep logstash
logstash.service disabled
[root@l ~]# systemctl restart logstash.service #### Restart
2.3. Errors encountered
[root@l opt]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
/usr/share/logstash/vendor/jruby/bin/jruby: Line 401: /usr/bin/java: No such file or directory
Unable to install system startup script for Logstash.
Solution
ln -s /usr/local/java/bin/java /usr/bin/java
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
3, configuration
chown -R logstash /etc/logstash/conf.d
touch /opt/logstash/messages
chown -R logstash /opt/logstash
chown -R logstash /opt/logstash/messages
chown -R logstash /var/log/messages
Shipper configuration file (logstash_shipper.conf)
vim logstash_shipper.conf
path => "/var/log/messages"
start_position => "beginning"
sincedb_path => "/dev/null"
Indexer configuration file (logstash_indexer.conf) Note: This
configuration file must be re-node node, otherwise the two output will
repeat the output log, plus the redis cache will be infinite output.
vim logstash_indexer.conf
if [type] == "messages" {
index => "messages-%{+YYYY-MM-dd}"
4, test
cd /usr/share/logstash/bin/
./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
[root@l bin]# ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
5, start
systemctl start logstash.service
systemctl enable logstash.service