Friday, August 28, 2020

show composer file current version

Composer show is the command that will provide the list of packages i.e. installed by you or your team member.

 

command is

 

 "composer show -i"

Tuesday, August 25, 2020

ELK enterprise application - elk quick build - logstash

 

ELK enterprise application - elk quick build - logstash

 

1, install JDK
elasticsearch, the operation of logstash depends on the java environment.
Download and unzip the jdk binary package.

  1. tar xf jdk-8u144-linux-x64.tar.gz -C /usr/local
  2. mv /usr/local/jdk1.8.0_144 /usr/local/java
  3. cd ~

Configure the java environment variable.
Add the following at the end of the ~/.bashrc file:

  1. export JAVA_HOME=/usr/local/java
  2. export JRE_HOME=$JAVA_HOME/jre
  3. export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/bin/tools.jar:$JRE_HOME/lib
  4. export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

Make the configuration take effect.

source ~/.bashrc

2, install Logstash

It is recommended that the Linux class server download the rmp package installation.
2.1. Download the logstash installation package

  1. touch /etc/default/logstash
  2. ln -s /usr/local/java/bin/java /usr/bin/java
  3. rpm -ivh logstash-6.2.4.rpm
  4. cd ~

2.2. Configure systemd to start

When installing rpm, the configuration file for creating the startup script is /etc/logstash/startup.options

/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

Note: When the script fails to start, you can create your own startup script.

  1. [root@l ~]# cat /etc/systemd/system/logstash.service
  2. [Unit]
  3. Description=logstash
  4. [Service]
  5. Type=simple
  6. ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
  7. ExecStop=/bin/kill -s QUIT $MAINPID
  8. ExecReload=/bin/kill -s HUP $MAINPID
  9. WorkingDirectory=/usr/share/logstash/bin
  10. [Install]
  11. WantedBy=multi-user.target
  12. [root@l ~]# systemctl daemon-reload #####Update
  13. [root@l ~]#
  14. [root@l ~]# systemctl list-unit-files |grep logstash
  15. logstash.service                              disabled
  16. [root@l ~]#
  17. [root@l ~]# systemctl restart logstash.service #### Restart

 

2.3. Errors encountered

[root@l opt]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
/usr/share/logstash/vendor/jruby/bin/jruby: Line 401: /usr/bin/java: No such file or directory
Unable to install system startup script for Logstash.

Solution

  1. ln -s /usr/local/java/bin/java /usr/bin/java
  2. /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

3, configuration

  1. cd /etc/logstash/conf.d/
  2. chown -R logstash /etc/logstash/conf.d
  3. mkdir /opt/logstash
  4. touch /opt/logstash/messages
  5. chown -R logstash /opt/logstash
  6. chown -R logstash /opt/logstash/messages
  7. chown -R logstash /var/log/messages

Shipper configuration file (logstash_shipper.conf)

 
  1. vim logstash_shipper.conf
  2. ###########################################3
  3. input{
  4.   file{
  5.       type => "messages"
  6.       path => "/var/log/messages"
  7.       start_position => "beginning"
  8.       sincedb_path => "/dev/null"
  9.   }
  10. }
  11. output{
  12.    if [type] == "messages"{
  13.       redis{
  14.           host => "10.0.0.132"
  15.           data_type => "list"
  16.           key => "messages"
  17.           port => 6379
  18.           db => 2
  19.           password => "123456"
  20.       }
  21.   }
  22. }

Indexer configuration file (logstash_indexer.conf) Note: This configuration file must be re-node node, otherwise the two output will repeat the output log, plus the redis cache will be infinite output.

 
  1. vim logstash_indexer.conf
  2. ######################################
  3. input{
  4.   redis{
  5.       host => "10.0.0.132"
  6.       data_type => "list"
  7.       key => "messages"
  8.       password => "123456"
  9.       db => 2
  10.   }
  11. }
  12. output{
  13.    if [type] == "messages" {
  14.       elasticsearch{
  15.           hosts => ["10.0.0.130"]
  16.           index => "messages-%{+YYYY-MM-dd}"
  17.       }
  18.   }
  19. }

4, test

 
  1. cd /usr/share/logstash/bin/
  2. ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
  3. [root@l bin]# ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
  4. Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
  5. Configuration OK

5, start

 
  1. systemctl start logstash.service
  2. systemctl enable logstash.service

 

Friday, July 3, 2020

mongo dump restore

Following is the command for the restore of mongo database 

 mongorestore -u username -p password --host localhost --authenticationDatabase databasename -d /tmp/mongobkp/


Sunday, May 3, 2020

JENKIN CHEATSHEET


Openshift cheatsheet

Openshift Login and Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#login with a user
oc login https://192.168.99.100:8443 -u developer -p developer
 
#login as system admin
oc login -u system:admin
 
#User Information
oc whoami
 
#View your configuration
oc config view
 
#Update the current context to have users login to the desired namespace:
oc config set-context `oc config current-context` --namespace=

Basic Commands

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#Use specific template
oc new-app https://github.com/name/project --template=
 
#New app from a different branch
oc new-app --name=html-dev nginx:1.10~https://github.com/joe-speedboat/openshift.html.devops.git#mybranch
 
#Create objects from a file:
oc create -f myobject.yaml -n
 
#Create or merge objects from file
oc apply -f myobject.yaml -n
 
#Update existing object
oc patch svc mysvc --type merge --patch '{"spec":{"ports":[{"port": 8080, "targetPort": 5000 }]}}'
 
#Monitor Pod status
watch oc get pods
 
#show labels
oc get pods --show-labels 
 
#Gather information on a project's pod deployment with node information
$ oc get pods -o wide
 
#Hide inactive Pods
oc get pods --show-all=false
 
#Display all resources 
oc get all,secret,configmap
 
#Get the Openshift Console Address
oc get -n openshift-console route console
 
#Get the Pod name from the Selector and rsh in it
POD=$(oc get pods -l app=myapp -o name)
oc rsh -n $POD
 
#exec single command in pod
oc exec $POD $COMMAND
 
#Copy file from myrunning-pod-2 path in the current location
oc rsync myrunning-pod-2:/tmp/LogginData_20180717220510.json .
 
#Read resource schema doc
oc explain dc

Image Streams

1
2
3
4
5
6
7
8
#List available IS for openshift project
oc get is -n openshift
 
#Import an image from an external registry
oc import-image --from=registry.access.redhat.com/jboss-amq-6/amq62-openshift -n openshift jboss-amq-62:1.3 --confirm
 
#List available IS and templates
oc new-app --list

WildFly application example

1
2
3
oc create -f https://raw.githubusercontent.com/wildfly/wildfly-s2i/wf-18.0/imagestreams/wildfly-centos7.json
oc new-app wildfly~https://github.com/fmarchioni/ocpdemos --context-dir=wildfly-basic --name=wildfly-basic
oc expose svc/wildfly-basic

Create app from a Project with Dockerfile

1
2
3
4
5
6
7
8
9
oc new-build --binary --name=mywildfly -l app=mywildfly
 
oc patch bc/mywildfly -p '{"spec":{"strategy":{"dockerStrategy":{"dockerfilePath":"Dockerfile"}}}}'
     
oc start-build mywildfly --from-dir=. --follow
 
oc new-app --image-stream=mywildfly
     
oc expose svc/mywildfly

Nodes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#Get Nodes lits
oc get nodes
 
#Check on which Node your Pods are running
oc get pods -o wide
 
#Schedule an application to run on another Node
oc patch dc  myapp -p '{"spec":{"template":{"spec":{"nodeSelector":{"kubernetes.io/hostname": "ip-10-0-0-74.acme.compute.internal"}}}}}'
 
#List all pods which are running on a Node
oc adm manage-node node1.local --list-pods
 
#Add a label to a Node
oc label node node1.local mylabel=myvalue
 
#Remove a label from a Node
oc label node node1.local mylabel-

Storage

1
2
3
4
5
6
7
8
9
10
#create a PersistentVolumeClaim (+update the DeploymentConfig to include a PV + update the DeploymentConfig to attach a volumemount into the specified mount-path)
  
oc set volume dc/file-uploader --add --name=my-shared-storage \
-t pvc --claim-mode=ReadWriteMany --claim-size=1Gi \
--claim-name=my-shared-storage --claim-class=ocs-storagecluster-cephfs \
--mount-path=/opt/app-root/src/uploaded \
-n my-shared-storage
 
#List storage classes
oc -n openshift-storage get sc

Build

1
2
3
4
5
6
7
8
#Manual build from source 
oc start-build ruby-ex
 
#Stop a build that is in progress  
oc cancel-build
 
#Changing the log level of a build:
oc set env bc/my-build-name BUILD_LOGLEVEL=[1-5]

Deployment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#Manual deployment
$ oc rollout latest ruby-ex
 
#Pause automatic deployment rollout
oc rollout pause dc $DEPLOYMENT
 
# Resume automatic deployment rollout
oc rollout resume dc $DEPLOYMENT
 
#Define resource requests and limits in DeploymentConfig
oc set resources deployment nginx --limits=cpu=200m,memory=512Mi --requests=cpu=100m,memory=256Mi
 
#Define livenessProve and readinessProve in DeploymentConfig
oc set probe dc/nginx --readiness --get-url=http://:8080/healthz --initial-delay-seconds=10
oc set probe dc/nginx --liveness --get-url=http://:8080/healthz --initial-delay-seconds=10
 
#Define Horizontal Pod Autoscaler (hpa)
oc autoscale dc $DC_NAME --max=4 --cpu-percent=10

Routes

1
2
3
4
5
#Create route
 $ oc expose service ruby-ex
 
#Read the Route Host attribute
oc get route my-route -o jsonpath --template="{.spec.host}"

Services

1
2
3
4
5
#Make a service idle. When the service is next accessed will automatically boot up the pods again:
$ oc idle ruby-ex
 
#Read a Service IP
oc get services rook-ceph-mon-a --template='{{.spec.clusterIP}}'

Clean up resources

1
2
3
4
5
6
7
8
9
10
11
12
13
#Delete all resources
oc delete all --all
 
#Delete resources for one specific app
$ oc delete services -l app=ruby-ex
$ oc delete all -l app=ruby-ex
 
#CleanUp old docker images on nodes
#Keeping up to three tag revisions 1, and keeping resources (images, image streams and pods) younger than sixty minutes:
oc adm prune images --keep-tag-revisions=3 --keep-younger-than=60m
 
#Pruning every image that exceeds defined limits:
oc adm prune images --prune-over-size-limit

Troubleshooting

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#Check status of current project   
oc status
 
#Get events for a project
oc get events --sort-by='{.lastTimestamp}'
 
# get the logs of the myrunning-pod-2-fdthn pod
oc logs myrunning-pod-2-fdthn
# follow the logs of the myrunning-pod-2-fdthn pod
oc logs -f myrunning-pod-2-fdthn
# tail the logs of the myrunning-pod-2-fdthn pod
oc logs myrunning-pod-2-fdthn --tail=50
 
#Check the integrated Docker registry logs:
oc logs docker-registry-n-{xxxxx} -n default | less
 
#run cluster diagnostics
oc adm diagnostics

Security

1
2
3
4
5
#Create a secret from the CLI and mount it as a volume to a deployment config:
oc create secret generic oia-secret --from-literal=username=myuser
 --from-literal=password=mypassword
oc set volumes dc/myapp --add --name=secret-volume --mount-path=/opt/app-root/
 --secret-name=oia-secret

Manage user roles

1
2
3
oc adm policy add-role-to-user admin oia -n python
oc adm policy add-cluster-role-to-user cluster-reader system:serviceaccount:monitoring:default
oc adm policy add-scc-to-user anyuid -z default

Misc commands

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#Manage node state
oc adm manage node --schedulable=false
 
#List installed operators
oc get csv
 
#Export in a template the IS, BC, DC and SVC
oc export is,bc,dc,svc --as-template=app.yaml
 
#Show user in prompt
function ps1(){
   export PS1='[\u@\h($(oc whoami -c 2>/dev/null|cut -d/ -f3,1)) \W]\$ '
}
 
#backup openshift objects
 
oc get all --all-namespaces --no-headers=true | awk '{print $1","$2}' | while read obj
do
  NS=$(echo $obj | cut -d, -f1)
  OBJ=$(echo $obj | cut -d, -f2)
  FILE=$(echo $obj | sed 's/\//-/g;s/,/-/g')
  echo $NS $OBJ $FILE; oc export -n $NS $OBJ -o yaml > $FILE.yml
done

Linux Commands Cheat Sheet in Black & White

Linux Commands Cheat Sheet in Black & White 

Keep this Linux command cheat sheet on your desk printed, I am sure you will learn them quickly and will be a Linux expert very soon. We have added had both pdf and image (png) format of the cheat sheet.
Command are categorized into different sections according to its usage. We have designed the command in white color with black background as we often use on Linux shell. We have added bit color for attraction.
We have grouped Linux commands in the below sections for better understanding.

Linux command cheat sheet

  • System
  • Hardware
  • Users
  • File Commands
  • Process Related
  • File Permission
  • Network
  • Compression / Archives
  • Install Packages
  • Install Source
  • Search
  • Login
  • File Transfer
  • Directory Traverse


Download your linux commands cheat sheet in pdf format which is updated in 2019. Please keep us posted if you have any suggestions or if you find any command that we missed out.
If you are looking to print Linux commands cheat sheet in A4 size paper its available for download.

1) System

uname  Displays  Linux system information
uname -rDisplays  kernel release information
uptimeDisplays how long the system has been running including load average
hostnameShows the system hostname
hostname -iDisplays the IP address of the system
last rebootShows system reboot history
dateDisplays current system date and time
timedatectlQuery and change the System clock
calDisplays the current calendar month and day
wDisplays currently  logged in users in the system
whoamiDisplays who you are logged in as
finger usernameDisplays information about the user

2) Hardware

dmesgDisplays bootup messages
cat /proc/cpuinfoDisplays more information about CPU e.g model, model name, cores, vendor id
cat /proc/meminfoDisplays more information about hardware memory e.g. Total and Free memory
lshwDisplays information about system's hardware configuration
lsblkDisplays block devices related information
free -mDisplays free and used memory in the system (-m flag indicates memory in MB)
lspci -tvDisplays PCI devices in a tree-like diagram
lsusb -tvDisplays USB devices in a tree-like diagram
dmidecodeDisplays hardware information from the BIOS
hdparm -i /dev/xdaDisplays information about disk data
hdparm -tT /dev/xda <:code>Conducts a read speed test on device xda
badblocks -s /dev/xdaTests  for unreadable blocks on disk

3) Users

idDisplays the details of the active user e.g. uid, gid, and groups
lastShows the last logins in the system
whoShows who is logged in to the system
groupadd "admin" Adds the group 'admin'
adduser "Sam" Adds user Sam
userdel "Sam"Deletes user Sam
usermodUsed for changing / modifying user information

4) File Commands

ls -alLists files - both regular &  hidden files and their permissions as well.
pwdDisplays the current directory file path
mkdir 'directory_name'Creates a new directory
rm file_name Removes a file
rm -f filenameForcefully removes a file
rm -r directory_nameRemoves a directory recursively
rm -rf directory_nameRemoves a directory forcefully and recursively
cp file1 file2Copies the contents of file1 to file2
cp -r dir1 dir2Recursively Copies dir1 to dir2. dir2 is created if it does not exist
mv file1 file2Renames file1 to file2
ln -s /path/to/file_name   link_nameCreates a symbolic link to file_name
touch file_nameCreates a new file
cat > file_namePlaces standard input into a file
more file_nameOutputs the contents of a file
head file_nameDisplays the first 10 lines of a file
tail file_nameDisplays the last 10 lines of a file
gpg -c file_nameEncrypts a file
gpg file_name.gpgDecrypts a file
wcPrints the number of bytes, words and lines in a file
xargsExecutes commands from standard input

5) Process Related

psDisplay currently active processes
ps aux | grep 'telnet'Searches for the id of the process 'telnet'
pmapDisplays memory map of processes
top Displays all running processes
kill pidTerminates process with a given pid
killall procKills / Terminates all processes named proc
pkill process-nameSends a signal to a process with its name
bgResumes suspended jobs in the background
fgBrings suspended jobs to the foreground
fg n job n to the foreground
lsofLists files that are open by processes
renice 19 PIDmakes a process run with very low priority
pgrep firefoxfind Firefox process ID
pstreevisualizing processes in tree model

6) File Permission

chmod octal filename         Change file permissions of the file to octal
 
Example 
chmod 777 /data/test.c       Set rwx permissions to owner, group and everyone (everyone else who has access to the server)
chmod 755 /data/test.c       Set rwx to the owner and r_x to group and everyone
chmod 766 /data/test.c       Sets rwx for owner, rw for group and everyone
chown owner user-file         Change ownership of the file
chown owner-user:owner-group file_name       Change owner and group owner of the file
chown owner-user:owner-group directory  Change owner and group owner of the directory

7) Network

ip addr show                   Displays IP addresses and all the network interfaces
ip address add 192.168.0.1/24 dev eth0    Assigns IP address 192.168.0.1 to interface eth0
ifconfig                             Displays IP addresses of all network interfaces
ping  host                       ping command sends an ICMP echo request to establish a connection to server / PC
whois domain                  Retrieves more information about a domain name
dig domain                       Retrieves DNS information about the domain
dig -x host                    Performs reverse lookup on a domain
host google.com          Performs an IP lookup for the domain name
hostname -i                     Displays local IP address
wget file_name             Downloads a file from an online source
netstat -pnltu     Displays all active listening ports

8) Compression/Archives

tar -cf home.tar home<:code>Creates archive file called 'home.tar' from file 'home'
tar -xf files.tar              Extract archive file 'files.tar'
tar -zcvf home.tar.gz source-folder    Creates gzipped tar archive file from the source folder
gzip file Compression a file with .gz extension

9) Install Packages

rpm -i pkg_name.rpm            Install an rpm package
rpm -e pkg_name                     Removes an rpm package
dnf install pkg_nameInstall package using dnf utility

10) Install Source (Compilation)

./configureChecks your system for the required software needed to build the program. It will build the Makefile containing the instructions required to effectively build the project
makeIt reads the Makefile to compile the program with the required operations. The process may take some time, depending on your system and the size of the program
make installThe command installs the binaries in the default/modified paths after the compilation

11) Search

grep 'pattern' files                           Search for a given pattern in files
grep -r pattern dir                             Search recursively for a pattern in a given directory
locate file                                            Find all instances of the file
find /home/ -name "index"              Find file names that begin with 'index' in /home folder
find /home -size +10000kFind files greater than 10000k in the home folder

12) Login

ssh user@host                                        Securely connect to host as user
ssh -p port_number user@host      Securely connect to host using a specified port
ssh host                                                Securely connect to the system via SSH default port 22
telnet host Connect to host via telnet default port 23

13) File Transfer

scp file1.txt server2/tmp           Securely copy file1.txt to server2 in /tmp directory
rsync -a /home/apps  /backup/ Synchronize contents in /home/apps directory with /backup  directory

14) Disk Usage

df  -h                           Displays free space on mounted systems
df  -i                          Displays free inodes on filesystems
fdisk  -l                    Shows disk partitions, sizes, and types
du  -sh                        Displays disk usage in the current directory in a human-readable format
findmnt                      Displays target mount point for all filesystems
mount device-path mount-pointMount a device

15) Directory Traverse

cd ..             Move up one level in the directory tree structure
cd                 Change directory to $HOME directory
cd /test Change directory to /test directory

 

How to solve mysql ERROR 1118 (42000) Row size too large

  I had this issue with MYSQL 5.7 . The following worked althoug...