Tuesday, August 30, 2011

Unix commands for Developers


This article is in continuation to -


Unix Grep Commands for Developers
Unix Find Commands for Developers
Unix Network Commands for Developers
Unix File Permissions


Please make sure you get an understanding of the concepts in these articles before reading further.


###########################################################


list all the links in a directory


ls -ltR | grep ">"


###########################################################


create a read only file


touch filename
chmod 400 filename


if file is already created and we do 
touch filename


it changes the modified timestamp to now.


###########################################################


Running a background process (Deamon)- use '&' in end of command


./import_data.sh &


java MyClass &


###########################################################


Dumping the Output of a process to a file


java MyClass > logFile.txt


./import_data.sh > log.txt


note: space between command and file name ' > log.txt'


./import_data.sh param1 param2 > log.txt &


nohup 
- command to ignore the HUP (hangup) signal, enabling the command to keep running after the user who issues the command has logged out. The HUP (hangup) signal is by convention the way a terminal warns depending processes of logout.


nohup ./import_data.sh param1 param2 > log.txt &


###########################################################


To see if process is running


ps -eaf | grep someTextInProcess


ps -eaf | grep import_
ps -eaf | grep myClass


###########################################################


History of cmds executed


history


Search for specific commands in history


history | grep "somethingFromPastCmd"


###########################################################


disk free


display the amount of available disk space for filesystems, on which the invoking user has appropriate read access


df


df [-k] [file...]


k = kilo
h = human redable. It is in K,M,G. K=kilo, M=mega,G=Giga
file = amount of free space of the file system containing the specified file
which directory is taking how much space


###########################################################


Disk Usage


estimate file space usage—space used under a particular directory or files on a file system.


gives the total size of current dir


du –sh


gives the total size, folder wise and file wise (files present directly in that directory)


du -sk *


Disk usage of all subdirectories and files including hidden files within the current directory (sorted by filesize) :


$ du -sk .[!.]* *| sort -n


Disk usage of all subdirectories and files including hidden files within the current directory (sorted by reverse filesize) :


$ du -sk .[!.]* *| sort -nr


###########################################################


Synbolic Links


-------
Create
-------


-
Soft link -
-
If the original program, file, or directory is renamed, moved, or deleted, the soft link is broken


ln -s directoryTo linkName


-
Hard link -
-
Unix Hard links are pointers to programs and files, but NOT directories
If the original program or file is renamed, moved, or deleted, the hard link is NOT broken
Hard links in UNIX cannot span disk drives,


ln myfile mylink


-------
Update
-------


ln -nsf newDirectory existingLinkName


-------
Delete
-------


rm -rf linkName


###########################################################


Zombie Process: 


The process is dead but have not been removed from the process table.


If you do 'ps', the child will have a 'Z' in its status field to indicate this


###########################################################


export JAVA_HOME=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
export PATH=$PATH:$JAVA_HOME


set it in ~/.bashrc OR ~/.prifile


###########################################################


32 bit or 64 bit ?
"uname -a"


###########################################################


Substitute - Find and replace


Replace words in a file


sed s/OldWord/NewWord/g fileName.


s stands for substitute


sed s/day/night/ oldFileName > newFileName




Using & as the matched string


echo "123 abc" | sed 's/[0-9]*/& &/'
123 123 abc


sed 's/[a-z]*/(&)/' oldFileName > newFileName


sed s/[0-9]*/'& & &'/g myfile.txt


d command to delete lines 
delete lines that are either blank or only contain spaces:
sed '/^ *$/d' inputFileName 




###########################################################


ftp


http://www.cs.colostate.edu/helpdocs/ftp.html


To connect your local machine to the remote machine, type


ftp machinename


When you enter your own loginname and password for the remote machine, it returns the prompt


ftp>


and permits you access to your own home directory on the remote machine.


commands


-------------------
to delete (remove) a file in the current remote directory (same as rm in UNIX)


delete fileName


-------------------


to copy one file from the remote machine to the local machine


get fileName


-------------------


to make a new directory within the current remote directory


mkdir dir


-------------------


to copy multiple files from the remote machine to the local machine;
you are prompted for a y/n answer before transferring each file


mget ABC DEF
mget *


-------------------
to copy one file from the local machine to the remote machine


put fileName


-------------------
mput ABC DEF


to copy multiple files from the local machine to the remote machine;
you are prompted for a y/n answer before transferring each file
-------------------


quit   to exit the FTP environment (same as bye)




###########################################################


NUMBER OF LINES IN A FILE


wc -l fileName


SHOW ONLY FEW LINES FROM FILE


less fileName


NUMBER OF SPECIFIC WORDS IN FILE


grep "word" <filename> | wc -l
grep www.google.com/bot.html access_log | wc -l


- put quotes around the word if it has spaces


###########################################################


=== copy files to remote host ===


syntax-
scp files user@remote.hostname:/path_to_copy_file


examples-


a) Copy from current to remote -


scp *.txt user@remote.server.com:/home/user/
 -This will copy all files with .txt extension to the directory /home/user in the remote.server.com host


b) Copy from remote to current  -


scp -r miguel@10.1.2.2:/home/myfolder/ ashish@10.1.2.3:/home/myfolder/
- This is going to recursively copy all files from myfolder's directory on 10.1.2.2 host to his myfolder directory in 10.1.2.3 host.


-r is to copy the entire directory


Sometimes while copying files from prod to dev does not work when u are logged in prod and try (a) - due to firewall/security settings. Then (b) helps by logging on local dev and trying to copy from remote.


Note : To use this command you need to have open-ssh installed in the hosts.


###########################################################


vi myFile.txt


- before entering data, enable insert mode - by pressing I
- to save after edit --- esc + !wq + enter
- to delete a line --- esc + press D twice for one line


SEE FIRST OCCURANCE OF A WORD IN FILE


vi file
/<word>


SEE LAST OCCURANCE OF A WORD IN FILE


vi file
?<word>


after the word is found, go to next word
press n


Go to end of file


G


Capital G


/s/oldWord/newWord/g - replace all occurrences



/s/oldWord/newWord - replace first occurrenc, then have to press n to go to next and replace


###########################################################


Sort


ls -s | sort -n


the the contents in a file alphabaticallt


sort myfile


Sort Decsending
sort -r myfile


###########################################################
cat 


to display file or concatenate files
cat filename


###########################################################


JAVA


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- COMPILE ---
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


javac -cp . Test.java




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- RUN ---
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


- you can add entire directory in classpath -- ../lib;
- only specify classname like MyJavaClass.. not the extention like MyJavaClass.class
- space separated entries after classname are String[] args that get supplied as params to main() method
- if have folder structure in java packacge like myfolder.mypackage.MyJavaClass. Then maintain the same physical folder/directory structure in my_job.jar.




--------------------
~~~ In WINDOWS ~~~~
--------------------


java -Xms2G -Xmx4G -cp .;../lib/mysql-connector-java-5.1.0-bin.jar;../lib/mq.jar;my_job.jar;../lib; myfolder.mypackage.MyJavaClass param1 param2


- all items in classpath are semi-colon separated


--------------------
~~~ In UNIX ~~~~
--------------------




java -Xms2G -Xmx4G -cp .:../lib/mysql-connector-java-5.1.0-bin.jar:../lib/mq.jar::my_job.jar:../lib: myfolder.mypackage.MyJavaClass param1 param2


- all items in classpath are colon separated.


###########################################################


2 comments: