Sunday, July 24, 2016

nix-tools and commands every programmer must know

Most common nix-tools

cd /var/some_directory
pwd - print current working directory

mkdir - make directory, can be used with -p flag to create recursively nonexistent parent dirs

ls -l - list all dirs and files with their permissions and modified time / if U need to print all hidden files + human readable weights of a files You may use - ls -lah

nano - the simplest editor to fast edit files

vim - more hard to learn then nano, but in practice sometimes U will get in situations where something opened by this editor and U have no choice - learn how to edit text and close it, at least.

| - this symbol is pipe, that redirects the output to any other stream

> write line to file
>> append line to file

echo - print everything to output, ex.: U can use something like this echo "test" >> /var/log/my.log

cat - print text from a file to stdout 

grep - finds text in any file(s), output etc. this tool is tremendously useful, especially in combination with others, later You will see 

kill / killall - for the 1st U need to provide the pid to kill (sometimes with -9 to send SIGKILL to running program ), for the 2nd the name of a program

mc - Midnight Commander, nothing to add just install it by default to do work much faster and transfer anything over the net

htop - install this by default either, to monitor LA, CPU, HEAP/SWAP etc

df -h - find out disk space in human readable format

find / locate - finds a file

which - which program used providing absolute path

netstat - status of connections to server

wget / curl - tools to make requests over the network with many options including for ex.: ssl connections etc

ping - pings the host, checking if it is alive

telnet - test network connection for IP on any PORT

traceroute - displaying the route and measuring transit delays of packets

Miscellaneous and combination of nix commands/tools
(kids may  go to sleep)

cat filename | grep sometext - become acquainted this is Your life saver on high-load systems when You try to pick needle in a haystack.

ps aux | grep program_name - find a special running process 

history | grep %command part% - finds a command previously executed

tail -fn 20 filename - show last 20 lines of a file and proceed to output last written (appended)

head -fn 25 filename - show first 25 lines of a file and proceed to output last written (prepended)

netstat -tlpn - show active internet connections

nmap -p 5123 94.240.126.22 - test PORT for particular IP

Rock `n` Rolling nix commands/tools
(older generation still here?)

du -csh * | sort -n - find the most heavy folders and sort them 

find /path/to/files* -mtime +5 -exec rm {} \; - delete files older then 5 days

find folder -depth -type f -atime +7 -delete - find files that have a time of modification older then 7 days

find -vc 'special*' | wc -l - find files started with special and print counter

chmod +x $(find /var/www/someproject/ -type d) - chmod only for dirs

netstat -na | grep 3306 | wc -l - count all processes of mysql-server 

grep --include=\*.{c,h} -rnw 'directory' -e "pattern" - search for pattern in directory, including only files with .c/.h extensions
grep --exclude=*.o -rnw 'directory' -e "pattern" - search for pattern in directory, excluding only files with .o extensions

Detect DDoS or frequent connections in the network (graphical way):
netstat -an|grep ESTABLISHED|awk '{print $5}'|awk -F: '{print $1}'|sort|uniq -c|awk '{ printf("%s\t%s\t",$2,$1); for (i = 0; i < $1; i++) {printf("*")}; print ""}'

Detect DDoS or frequent connections in the access.log files:
head -n 1000000 /var/log/nginx/yourprojectexample.com.access.log|cut -f 1 -d ' '|sort|uniq -c|sort -nr|more

Examples of output:
 953558 213.X.X.X
   6218 37.X.2.X
   2400 46.X.36.X
   1128 109.X.124.X
........

There is an extra one - my favorite, if U`ll ever be in critical situation and need to do things fast with previously performed action - try this:
!ops - this will search in a history for "%ops%" wild-card and run, as an example U might want to connect to mysql-server with command like mysql -u username -h somehost -pPassword db_name by just executing: !mysql