> For the complete documentation index, see [llms.txt](https://davin-hong3.gitbook.io/d/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://davin-hong3.gitbook.io/d/programming/hackerrank/bash-linux-shell.md). # Bash (Linux Shell) ### Cut #1 - 4 (easy cuts) {% embed url="" %} ![](/files/kgrWxGVARhfuzwVCknWl) ```bash # OVERVIEW ## 1. Cut third field of each line - Output : three \n gamma cut -f 3 data.txt ## 2. Cut third field and every field after it, omit first 2 fields cut -f 3- data.txt >> three four five gamma delta epsilon ## 3. 3rd char of every line in a file cut -c 3 file.txt ## 4. output first 3 char of every line cut -c 1-3 file.txt ## 5. output first 3 char and omit the rest / last 3 char cut -c -3 file.txt / cut -c 3- file.txt ## 6. Output the first field of the file /etc/passwd, where fields are ### delimited by a colon (':'). The first field of /etc/passwd is the username, ### so this command outputs every username in the passwd file. cut -d ':' -f 1 /etc/passwd ## 7. Otput 1st and 6th fields, delimited by colon, of any entry in the /etc/passwd file ### which specifies /bin/bash as the login shell. This command outputs the username ### and home directory of any user whose login shell is /bin/bash. grep '/bin/bash' /etc/passwd cut -d ':'-f 1,6 ## 8. Counting bytes cut -b 3-12 data.txt ## 9. To add delimiter to output (adding space in this case) cut -f 1,3 -d ':' --output-delimiter=' ' /etc/passwd ##10. To add tab delimiter for output (tab is special case, cause it is unprintable) cut -f 1,3 -d ':' --output-delimiter=$'\t' /etc/passwd #use $ to protect from shell ``` {% hint style="info" %} **/etc/passwd File** For instance, the file **/etc/passwd** contains information about each user on the system, one user per line, and each information field is delimited by a colon ("**:**"). For example, the line of **/etc/passwd** for the **root** user may look like this: ``` root:x:0:0:root:/root:/bin/bash ``` These fields contain the following information, in the following order, separated by a colon character: 1. Username 2. Password (shown as **x** if [encrypted](https://www.computerhope.com/jargon/e/encrypt.htm)) 3. User ID number or UID 4. Group ID number or GID 5. Comment field (used by the [finger](https://www.computerhope.com/unix/ufinger.htm) command) 6. [Home Directory](https://www.computerhope.com/jargon/h/homedir.htm) 7. [Shell](https://www.computerhope.com/jargon/s/shell.htm) {% endhint %} It is often used for parsing data from log files, csv files, and similar. Given a firewall's log file, for example, `cut` and `sort -u` can be used to create a sorted list of unique ip addresses. Add in `grep` to get a strong tool for carving out data. The following lines show how to extract data from a line of text. **Example below:** The `-d` flag sets the delimiter, space in this case, and the `-f` flag shows which column to return, . The column count starts at . In the next command, output from the first command is piped to a second command where the delimiter is a period and the column is . Finally, `cut` is used to extract the first character from the results of the second command. ```bash echo '0000 192.168.1.100 192.168.100.1' |cut -d ' ' -f 2 192.168.1.100 echo '0000 192.168.1.100 192.168.100.1' |cut -d ' ' -f 2 |cut -d '.' -f 4 100 echo '0000 192.168.1.100 192.168.100.1' |cut -d ' ' -f 2 |cut -d '.' -f 4|cut -c 1 1 ``` The above is for command line inputs. **HackerRank Task :** Read N lines of input, then output the 3rd character of each input. (etc. Input = character, output = a) ```bash #!/bin/bash while read line # while there is input, line is th variable here, #read is the function to read input do echo $line | cut -c 3 done # to output characters 2nd and 7th while read line do echo $line | cut -c 2,7 done ``` ### Cut #5 - (tabs) 1. **No quote vs Double Quotes** When double quotes are used white space (tab space is a form of white space) is preserved. When double quotes aren't used tabspace isn't preserved. cut by default uses tabspace as a delimiter to extract the fields. If no tabspaces are present (and if delimiter option of cut hasn't been modified) then cut wont work. **2. IFS (Internal Field Separator)** It is used by the shell to determine how to do word splitting, i. e. how to recognize word boundaries. The default value for `IFS` consists of whitespace characters (to be precise: space, tab and newline). Each character can be a word boundary. ```bash #!/bin/bash ### TASKS ANSWER while read line do echo $line | cut -f1-3 ## METHOD 1. WILL NOT WORK ## VERSUS echo "$line" | cut -f1-3 ## METHOD 2. WORKS done ## USING IFS IFS="" # NO SPACES BETWEEN THE IFS AND = sign while read line; do echo -e $line | cut -f -3 done ``` ### Head & Tails Used to display first few lines of a file or last few lines of file ```bash head [filename] head -n 11 [filename] -> First 11 lines head -c 20 [filename] -> First 20 characters # if read from STDIN, ignore filename head -n 20 tail -n 20 # last twenty lines ## To get middle of text file (to get lines 12-22) head -n 22 | tail -n 11 # get first 22, then get last 11 of these 22 lines ``` ### tr (translate) Translate offers various ways that the *tr* command may be used for transforming or translating characters or character classes. It can also be used to delete, complement, remove or squeeze characters -c : complements the set of characters in string.i.e., operations apply to characters not in the given set -d : delete characters in the first set from the output. -s : replaces repeated characters listed in the set1 with single occurrence -t : truncates set1 For example:\ *'e' being transformed to 'E'* ``` $ echo "Hello" | tr "e" "E" HEllo ``` *Spaces being transformed to hyphens* ``` $ echo "Hello how are you" | tr " " '-' Hello-how-are-you ``` *Digits (numerals) being deleted* ``` $ echo "Hello how are you 1234" | tr -d [0-9] Hello how are you ``` Deleting lowercases a-z ``` tr -d [a-z] # flag of d for delete ``` Replace multiple spaces with single space ``` tr -s " " ```