Fanatical Solutions

Basically, screen allows you to create virtual terminals which are not connected to your actual xterms or console screens. You can then disconnect from a screen session and reconnect from somewhere else while preserving your shell or other running processes.

1. Set the screen binary (/usr/bin/screen) setuid root. By default, screen is installed with the setuid bit turned off, as this is a potential security hole.
2. The teacher starts screen in a local xterm, for example via screen -S SessionName. The -S switch gives the session a name, which makes multiple screen sessions easier to manage.
3. The student uses SSH to connect to the teacher’s computer.
4. The teacher then has to allow multiuser access in the screen session via the command Ctrl-a :multiuser on (all screen commands start with the screen escape sequence, Ctrl-a).
5. Next the teacher grants permission to the student user to access the screen session with Ctrl-a :acladd student where student is the student login ID.
6. The student can now connect to the teacher’s screen session. The syntax to connect to another user’s screen session is screen -x username/session.

alias ls='ls --color=none'

With that, all the pieces are in place: multiple users can share a screen session for any sort of command-line-based instruction. The teacher can at any time take control of the session by switching all other users to read-only access. Finally, you can turn on the logging facilities in screen to get an accurate and usable log of the entire session (or just portions of the session if you desire).

To make vi display line numbers, you need to set the number flag. To do so:

• Press the Esc key if you are currently in insert or append mode. 
   
• Press  :  (the colon). The cursor should reappear at the lower left corner of the screen next to a  :  prompt. 
   
• Enter the following command: set number
• A column of sequential line numbers will then appear at the left side of the screen. Each line number references the text located directly to the right.

To turn off line numbering, again follow the preceding instructions, except this time enter the following line at the  :  prompt: set nonumber

Normally, vi will forget the setting you’ve chosen once you’ve left the editor. You can, however, make either setting take effect automatically whenever you use vi on a particular account. To do this, place the appropriate set statement as a line in a file in your home directory named.exrc .

For a list of all current settings, at the  :  prompt, enter:set allFor a list of everything that you have set, at the  :  prompt, enter: set

Linux files are setup so access to them is controlled. There are three types of access:

• read
• write
• execute

Each file belongs to a specific user and group. Access to the files is controlled by user, group, and what is called other. The term, other, is used to refer to someone who is not the user (owner) of the file, nor is the person a member of the group the file belongs to. When talking about setting permissions for "other" users to use, it is commonly referred to as setting the world execute, read, or write bit since anyone in the world will be able to perform the operation if the permission is set in the other category.

File names and permission characters

File names can be up to 256 characters long with "-", "_", and "." characters along with letters and numbers.
When a long file listing is done, there are 10 characters that are shown on the left that indicate type and permissions of the file. File permissions are shown according to the following syntax example: drwerwerwe
There are a total of 10 characters in this example, as in all Linux files. The first character indicates the type of file, and the next three indicate read, write, and execute permission for each of the three user types, user, group and other. Since there are three types of permission for three users, there are a total of nine permission bits. The table below shows the syntax:

• Character 1 is the type of file: – is ordinary, d is directory, l is link.
• Characters 2-4 show owner permissions. Character 2 indicates read permission, character 3 indicates write permission, and character 4 indicates execute permission.
• Characters 5-7 show group permissions. Character 5=read, 6=write, 7=execute
• Characters 8-10 show permissions for all other users. Character 8=read, 9=write, 10=execute
• There are 5 possible characters in the permission fields. They are:
• r = read – This is only found in the read field.
• w = write – This is only found in the write field.
• x = execute – This is only found in the execute field.
• s = setuid – This is only found in the execute field.
• If there is a "-" in a particular location, there is no permission. This may be found in any field whether read, write, or execute field.

Examples

Type "ls -l" and a listing like the following is displayed:

Which means the following:

The fields are as follows:

• Type field: The first character in the field indicates a file type of one of the following:

1. d = directory
2. l = symbolic link
3. s = socket
4. p = named pipe
5. - = regular file
6. c= character (unbuffered) device file special
7. b=block (buffered) device file special

• Permissions are explained above.
• Links: The number of directory entries that refer to the file. In our example, there are four.
• The file’s owner in our example is George.
• The group the file belongs to. In our example, the group is team1.
• The size of the file in bytes
• The last modification date. If the file is recent, the date and time is shown. If the file is not in the current year, the year is shown rather than time.
• The name of the file.

Set User Identification Attribute

The file permissions bits include an execute permission bit for file owner, group and other. When the execute bit for the owner is set to "s" the set user ID bit is set. This causes any persons or processes that run the file to have access to system resources as though they are the owner of the file. When the execute bit for the group is set to "s", the set group ID bit is set and the user running the program is given access based on access permission for the group the file belongs to. The following command:

chmod +s myfile

sets the user ID bit on the file "myfile". The command:

chmod g+s myfile

sets the group ID bit on the file "myfile".

The listing below shows a listing of two files that have the group or user ID bit set.

-rws--x--x   1 root    root    14024 Sep  9 1999 chfn
-rwxr-sr-x   1 root   mail    12072 Aug 16 1999 lockfile

The files chfn and lockfile are located in the directory "/usr/bin". The "s" takes the place of the normal location of the execute bit in the file listings above. This special permission mode has no meaning unless the file has execute permission set for either the group or other as well. This means that in the case of the lockfile, if the other users (world execute) bit is not set with permission to execute, then the user ID bit set would be meaningless since only that same group could run the program anyhow. In both files, everyone can execute the binary. The first program, when run is executed as though the program is the root user. The second program is run as though the group "mail" is the user’s group.

For system security reasons it is not a good idea to set many program’s set user or group ID bits any more than necessary, since this can allow an unauthorized user privileges in sensitive system areas. If the program has a flaw that allows the user to break out of the intended use of the program, then the system can be compromised.

Directory Permissions

There are two special bits in the permissions field of directories. They are:

• s – Set group ID
• t – Save text attribute (sticky bit) – The user may delete or modify only those files in the directory that they own or have write permission for.

Save text attribute

The /tmp directory is typically world-writable and looks like this in a listing:

drwxrwxrwt   13 root     root         4096 Apr 15 08:05 tmp

Everyone can read, write, and access the directory. The "t” indicates that only the user (and root, of course) that created a file in this directory can delete that file.

To set the sticky bit in a directory, do the following:

chmod +t data

This option should be used carefully. A possible alternative to this is

• Create a directory in the user’s home directory to which he or she can write temporary files.
• Set the TMPDIR environment variable using each user’s login script.
• Programs using the tempnam(3) function will look for the TMPDIR variable and use it, instead of writing to the /tmp directory.

Directory Set Group ID

If the setgid bit on a directory entry is set, files in that directory will have the group ownership as the directory, instead of than the group of the user that created the file.

This attribute is helpful when several users need access to certain files. If the users work in a directory with the setgid attribute set then any files created in the directory by any of the users will have the permission of the group. For example, the administrator can create a group called spcprj and add the users Kathy and Mark to the group spcprj. The directory spcprjdir can be created with the set GID bit set and Kathy and Mark although in different primary groups can work in the directory and have full access to all files in that directory, but still not be able to access files in each other’s primary group.

The following command will set the GID bit on a directory:

chmod g+s spcprjdir

The directory listing of the directory "spcprjdir":

drwxrwsr-x 2 kathy spcprj 1674 Sep 17 1999 spcprjdir

The "s” in place of the execute bit in the group permissions causes all files written to the directory "spcprjdir" to belong to the group "spcprj" .

Examples

Below are examples of making changes to permissions:

Below are examples of making changes to owner and group:

Note: Linux files were displayed with a default tab value of 8 in older Linux versions. That means that file names longer than 8 may not be displayed fully if you are using an old Linux distribution. There is an option associated with the ls command that solves this problem. It is "-T". Ex: "ls al -T 30" to make the tab length 30.

Umask Settings

The umask command is used to set and determine the default file creation permissions on the system. It is the octal complement of the desired file mode for the specific file type. Default permissions are:

• 777 – Executable files
• 666 – Text files

These defaults are set allowing all users to execute an executable file and not to execute a text file. The defaults allow all users can read and write the file.

The permission for the creation of new executable files is calculated by subtracting the umask value from the default permission value for the file type being created. An example for a text file is shown below with a umask value of 022:

        666 Default Permission for text file
       -022 Minus the umask value
      —–
        644 Allowed Permissions

Therefore the umask value is an expression of the permissions the user, group and world will not have as a default with regard to reading, writing, or executing the file. The umask value here means the group the file belongs to and users other than the owner will not be able to write to the file. In this case, when a new text file is created it will have a file permission value of 644, which means the owner can read and write the file, but members of the group the file belongs to, and all others can only read the file. A long directory listing of a file with these permissions set is shown below.

-rw-r--r--   1 root     workgrp          14233 Apr  24 10:32 textfile.txt

A example command to set the umask is:

umask 022

The most common umask setting is 022. The /etc/profile script is where the umask command is usually set for all users.

Red Hat Linux has a user and group ID creation scheme where there is a group for each user and only that user belongs to that group. If you use this scheme consistently you only need to use 002 for your umask value with normal users.

1. The client sends the SQL statement to the server.

• The protocol is halfduplex, which means that at any given time the MySQL server can be either sending or receiving messages, but not both. It also means there is no way to cut a message short.
• The client sends a query to the server as a single packet of data. This is why the max_packet_size configuration variable is important if you have large queries. Once the client sends the query, it doesn’t have the ball anymore; it can only wait for results. The response from the server usually consists of many packets of data.When the server responds, the client has to receive the entire result set. It cannot simply fetch a few rows and then ask the server not to bother sending the rest. If the client needs only the first few rows that are returned, it either has to wait for all of the server’s packets to arrive and then discard the ones it doesn’t need, or disconnect ungracefully. Neither is a good idea, which is why appropriate LIMIT clauses are so important.Here’s another way to think about this: when a client fetches rows from the server, it thinks it’s pulling them. But the truth is, the MySQL server is pushing the rows as it generates them. The client is only receiving the pushed rows; there is no way for it to tell the server to stop sending rows. The client is “drinking from the fire hose,” so to speak.

2. The server checks the query cache. If there’s a hit, it returns the stored result from the cache; otherwise, it passes the SQL statement to the next step.

• Before even parsing a query, MySQL checks for it in the query cache, if the cache is enabled. This operation is a case sensitive hash lookup. If the query differs from a similar query in the cache by even a single byte, it won’t match, and the query processing will go to the next stage.
• If MySQL does find a match in the query cache, it must check privileges before returning the cached query. This is possible without parsing the query, because MySQL stores table information with the cached query. If the privileges are OK,
• MySQL retrieves the stored result from the query cache and sends it to the client, bypassing every other stage in query execution. The query is never parsed, optimized, or executed.

3. The server parses, preprocesses, and optimizes the SQL into a query execution plan.

• MySQL’s parser breaks the query into tokens and builds a “parse tree”from them. The parser uses MySQL’s SQL grammar to interpret and validate the query. For instance, it ensures that the tokens in the query are valid and in the proper order, and it checks for mistakes such as quoted strings that aren’t terminated. The preprocessor then checks the resulting parse tree for additional semantics that the parser can’t resolve. For example, it checks that tables and columns exist, and it resolves names and aliases to ensure that column references aren’t ambiguous.Next, the preprocessor checks privileges. This is normally very fast unless your server has large numbers of privileges.
• The parse tree is now valid and ready for the optimizer to turn it into a query execution plan. A query can often be executed many different ways and produce the same result. The optimizer’s job is to find the best option. MySQL uses a  ost-based optimizer, which means it tries to predict the cost of various execution plans and choose the least expensive. The unit of cost is a single random four-kilobyte data page read.

4. The query execution engine executes the plan by making calls to the storage engine API.

• he parsing and optimizing stage outputs a query execution plan, which MySQL’s query execution engine uses to process the query. The plan is a data structure; it is not executable byte-code, which is how many other databases execute queries. In contrast to the optimization stage, the execution stage is usually not all that complex: MySQL simply follows the instructions given in the query execution plan. Many of the operations in the plan invoke methods implemented by the storage engine interface, also known as the handler API. Each table in the query is represented by an instance of a handler. If a table appears three times in the query, for example, the server creates three handler instances. Though we glossed over this before, MySQL actually creates the handler instances early in the optimization stage. The optimizer uses them to get information about the tables, such as their column names and index statistics.

5. The server sends the result to the client.

• The final step in executing a query is to reply to the client. Even queries that don’t return a result set still reply to the client connection with information about the query, such as how many rows it affected.

Reference: http://adminlinux.blogspot.com/2009/06/mysql-query-execution-basics.html

First query cache is new and added in MySQL v4.x.x version only so if you are using old version of MySQL server it will not work.

When MySQL server recives a request it will parse it and retrives data from database/table and sent back to client browser. If same query request (in case of dynamic content) comes repeatedly and server will just sent them result from cache (thus saving disk I/O and other associated cost with each query).
Please note that when data stored in table is modified, any related cached entries in the query cache are flushed.

How to check the global variable from mysql prompt

mysql> show variables like 'query_cache_size';

mysql> show variables like 'query%';

How do I find out my MySQL query cache is working or not…Very simple, MySQL provides the stats of same just type following command at mysql> prompt:

mysql> show status like 'Qcache%';

mysql> show status like 'qc%';

How to clear the query cache

mysql> reset query cache;
 

Mtop is a little perl utility that shows in real time statistics of your MySQL server. It allows you to quickly identify problems related to mysql poor performance, using an interface similar to the normal ‘top’ command.

To install mtop on RHEL We  needed to first satisfy a dependency (perl-Curses) that is not available in the regular RHEL packages. The quickest way to install perl-Curses is to use a build rpm.

Depending from your RHEL version download the respective package, perl-Curses-1.13-1.1.el3.rf.i386.rpm for RHEL3 i386, or perl-Curses-1.13-1.2.el4.rf.x86_64.rpm for RHEL4 on x86_64.

Optional in case you don’t have already perl-DBI and perl-DBD, then install them using

yum install perl-DBD-MySQL perl-DBI


The other requirements (Getopt::Long, Net::Domain) are normally part of the base install and are already available

 To install on your Red Hat 5 i386 servers I did the following:

wget http://apt.sw.be/redhat/el4/en/i386/rpmforge/RPMS/mtop-0.6.6-1.2.el4.rf.noarch.rpm
wget http://apt.sw.be/redhat/el4/en/i386/rpmforge/RPMS/perl-Curses-1.27-1.el4.rf.i386.rpm
rpm -Uvh mtop-0.6.6-1.2.el4.rf.noarch.rpm perl-Curses-1.27-1.el4.rf.i386.rpm
 

 The RPMs installed the binary file in the /usr/bin directory. To use the program as the MySQL root user you can type on the command line:

mtop --dbuser=root --password=<MySQL root password>

For more information on using mtop you can type ‘man mtop’ on the command line.
 

Substitute either word1 or word2 with a new word using regular expression

In the following example, it will translate any occurrences of either good or nice will be replaced with awesome.You can also do substitution by specifying regular expression.

Original Text: Linux is good. Life is nice.

:%s/\(good\|nice\)/awesome/g

Translated Text: Linux is awesome. Life is awesome.

Following example does the substitution of hey or hi to hai. Please note that this does not do any substitution for the words ‘they’, ‘this’.

:%s/\<\(hey\|hi\)\>/hai/g

• \< – word boundary.
• \| – “logical or” (in this case hey or hi)

Interactive Find and Replace in Vim Editor

You can perform interactive find and replace using the ‘c’ flag in the substitute, which will ask for confirmation to do substitution or to skip it as explained below. In this example, Vim editor will do a global find the word ‘awesome’ and replace it with ‘wonderful’. But it will do the replacement only based on your input as explained below.

:%s/awesome/wonderful/gc
replace with wonderful (y/n/a/q/l/^E/^Y)?

• y – Will replace the current highlighted word. After replacing it will automatically highlight the next word that matched the search pattern
• n – Will not replace the current highlighted word. But it will automatically highlight the next word that matched the search pattern
• a – Will substitute all the highlighted words that matched the search criteria automatically.
• l – This will replace only the current highlighted word and terminate the find and replace effort.

Substituting all lines with its line number.

When the string starts with ‘\=’, it should be evaluated as an expression. Using the ‘line’ function we can get the current line number. By combining both the functionality the substitution does the line numbering of all lines.

:%s/^/\=line(".") . ". "/g

Note: This is different from the “:set number” where it will not write the line numbers into the file. But when you use this substitution you are making these line number available inside the file permanently.

Substituting special character with its equivalent value.

Substituting the ~ with $HOME variable value.

Original Text: Current file path is ~/test/

:%s!\~!\= expand($HOME)!g

Translated Text: Current file path is /home/ramesh/test/

You can use expand function to use all available predefined and user defined variables.

Syntax of the text substitution inside vim editor:

:[range]s[ubstitute]/{pattern}/{string}/[flags] [count]

Following are three possible flags.

1. [c] Confirm each substitution.
2. [g] Replace all occurrences in the line.
3. [i] Ignore case for the pattern.

Substitute all occurrences of a text with another text in the whole file

This is the basic fundamental usage of the text substitution inside Vi editor. When you want a specific text to be replaced with another text in the entire file then you can use the following sequence.

:%s/old-text/new-text/g

%s – specifies all lines. Specifying the range as ‘%’ means do substitution in the entire file.

g – specifies all occurrences in the line. With the ‘g’ flag , you can make the whole line to be substituted. If this ‘g’ flag is not used then only first occurrence in the line only will be substituted.

Substitution of a text with another text within a single line

When you want a specific text to be replaced with another text within a single line in a case insensitive manner. Specifying no range means, do substitution in the current line only. With the ‘i’ flag, you can make the substitute search text to be case insensitive.

:s/I/We/gi

Substitution of a text with another text within a range of lines

With the range, you can make only a range of line to be affected in the substitution. Specifying 1, 10 as range means, do substitution only in the lines 1 – 10.

:1,10s/helo/hello/g

Substitution of a text with another text by visual selection of lines

You can also select a specific lines by visually selecting those lines. Press CTRL + V in command mode, use navigation keys to select the part of the file you want to be substituted. Press ‘:’ which will automatically formed as :’<,’> Then you can use the normal substitute as

:'<,'>s/helo/hello/g

Substitution of a text with another text only the 1st X number of lines

Using count in substitution, If you specify the count N in the substitution then it means do substitution in N lines from the current position of the cursor. do substitution in 4 lines from the current line.

:s/helo/hello/g 4

Substitute only the whole word and not partial match

Let us assume that you want to change only the whole word ‘his’ to ‘her’ in the original text mentioned below. If you do the standard substitution, apart from changing his to her, it will also change This to Ther as shown below.

Standard Subsitution

Original Text: This is his idea

:s/his/her/g

Translated Text: Ther is her idea

Whole Word Subsitution

Original Text: This is his idea

:s/\<his\>/her/

Translated Text: This is her idea

Note: You should enclose the word with < and > , which will force the substitution to search only for the full word and not any partial match

find .  -type f -daystart -mtime -10 -exec ls -l {} \; | cpio -pdmv /output/data

find /var/web/public_html  -type f -daystart -mtime -10 | cpio -pdmv /output/data

I have given an explanation of each flag used in the above command as well.

• find , linux command
• . & /home/web1/public_html , specifies the directory to search, in this case the website document root
• -type f , will only search for files and not directories
• -daystart , will tell find to start with today date
• -mtime, tells find to include every thing that was modified on and before a twenty fours hour period
• | , will redirect the output to another binary, in this instance cpio
• cpio, Creates and un-creates archived cpio files. And also is capable of copying files to things other than a hard disk.
• -pdmv,
  1. -p pass trough (copy in and out)
  2. -d make dirs
  3. -m preserve modification time
  4. -v verbose
• /root/test, is the output directory

For moving the files we can use the below command

find . -name '*.bak' -exec mv '{}' backup_dir/ \;

This would move every file found into backup_dir/.

  find /home/web1/public_html  -type f -daystart -mtime -1 -exec ls -l {} \;
 | egrep -v "(xls$|jpg$|png$|gif$|tif$)" > /root/files_last_modified.txt

I have given an explanation of each flag used in the above command as well.

• /home/web1/public_html , specifies the directory to search, in this case the website document root
• -type f , will only search for files and not directories
• -daystart , will tell find to start with today date
• -mtime, tells find to include every thing that was modified on and before a twenty fours hour period
• -exec ls -l {}, will fun the ls -l command on each results
• | , will redirect the output to another binary, in this instance egrep
• The egrep flags will only display files that do not end in xls,jpg,png,gif, and tif.