Introduction
A process is the embodiment of a running application which may or may not consist of multiple threads. Processes have both attributes and well-delineated permissions. They must be in one of a number of enumerated states, the most common of which are running or sleeping. It is important to know when a process is running in user mode with limited privilege, or in kernel mode with enhanced priviledges, including more direct access to hardware. There are a number of different ways to create child processes and to set and modify their priorities.
Objectives
- Describe a process and its associated resources
- Distinguish between processes, programs and threads
- Understand process attributes, permissions and states
- know the difference between running in user and kernel modes
- Describe daemon processes
- Understand how new processes are forked
- Use nice and renice to set and modify priorities
Processes, Programs and Threads
A process is an executing program and associated resources, including environment, open files, signal handlers, etc. The same program may be executing more than once simultaneously and thus be responsible for multiple processes.
When we talk about multi-threaded process we mean a circumstance in which all the task or threads of execution share the same resources.
The init process
Is the first process that starts when the system get up and running. It is started as soon as the kernel is initialized and has mounted the root filesystem, its PID number is 1
Init will run until the system is shut down, it will be the last user process to be terminated at that point. It serves as the ancestral parent of all other user process, both directly and inderectly.
Processes
Is an instance of a program in execution, it may be in several states two of them are sleeping and running. Every process has a PID Process ID, they also have a PPID Parent Process ID and a PGID Process Group ID.
Every process has
- Program code
- Variables
- Environment
- Data
- File Descriptors
Init is usually the first user process to be ran on a system, and thus becomes the ancestor of all subsequent processes running on the system, except for those initiated directly from the kernel (which show up with [] around their name in a ps listing).
If a parent process dies before its child, the child's PPID becomes 1, the process is then adopted by init, however in recent version of Linux kernels using systemd, the PPID will be set to 2 which is the kthreadd, taking the role that once was from init process to adopt orphan processes.
For historical reasons the maximum PID number is limited to a 16 bit number or 32768, however we can change this value modifying
/proc/sys/kernel/pid_max
Since it may be inadequate for larger servers. As processes are created eventually they will reach pid_max at which point they will start again at PID=300
Process Attributes
- The program being executed
- Context (state)
- Permissions
- Associated resources
Every process is executing some program. At some point a process will take a snapshot of himself, with data like memory, the executing program, CPU registers, and other information. This is the context of the process.
In order for the kernel to manage the processes, it is important to keep this information to perform context switching.
Controlling Processes with ulimit
Command that displays or resets a number of resource limits associated with processes
$ ulimit -a
Used to
- Restrict capabilities so an individual user and/or process can not exhaust system resources, such as memory, cpu time or the maximum number of processes on the system.
- Expand capabilities so a process does not run into resource limits, for example a server handling many clients may find the default of 1024 open files makes its work impossible to perform.
Two kinds of limits
- Hard: The maximum value, set only by root user
- Soft : The current limiting value, which a user can modify but can not exceed the hard limit
One can set any particular limit by doing
$ ulimit [options] [limit]
as in
$ ulimit -n 1600
Which would increase the maximum number of file descriptors to 1600
Process Permissions and setuid
Every process has permissions based on which specific user invoked it. In addition, it may also have permissions based on who owns its program file.
- Programs which are marked with an s execute bit have a different effective user id than their real user id. These are referred to as setuid programs. They run with the user id of the user who owns the program
- Non setuid programs run with the permissions of the user who runs them
Note : Setuid programs owned by root can be a well known security problem.
- passwd program is an example of a setuid program, Any user can run it. Whe a user executes this program, the process must run with root permission in order to be able to update the write-restricted /etc/passwd and /etc/shadow files where the user passwords are maintained
More on Process States
Processes can be in one of several possible states, the main ones being
- Running
- The process is either currently executing on a CPU or CPU core or sitting in the run queue, eagerly awaiting a new time slice.
- Sleeping
- The process is waiting on a request (Usually I/O) that it has made and cannot proceed further until the request is completed.
- Stopped
- The process has been suspended. This state is commonly experienced when a programmer wants to examine the executing program's memory, CPU registers, flags, or other attributes. Once this is done the process may be resumed. Usually when the process is under debugging
- Zombie
- The process enters this state when it terminates, and no other process (usually the parent) has inquired about its exit state. A zombie process has released all of its resources except its exit state and its entry in the process table. If the parent of any process dies, the process is adopted by init (PID=1) or kthreadd (PID=2).
Execution Modes
At any given time a process (or any particular thread of a multi-threaded process) may be executing in either
- user mode
- system mode (or kernel mode)
What instructions can be executed depends on the mode and is enforced at the hardware, not software level.
The mode is not a state of the system, it is a state of the processor as in a multi-core or multi-CPU system each unit can be in its own individual state.
In Intel parlance, user mode is also termed Ring 3 and system mode is termed Ring 0
User Mode
Is a mode in which the process can run, that has limited priviledges. Each process executing in user mode has its own memory space, parts of which may be shared with other processes, except for the shared memory segments, a user process is not able to read or write into or from the memory space of any other process.
Even a process run by the root user as a setuid program runs in user mode except when jumping into a system call, and has only limited ability to access hardware.
Kernel Mode
In kernel mode the CPU has full access to all hardware on the system, including peripherals, memory, disks, etc. If an application needs access to these resources it must issue a sustem call, which causes a context switch from user mode to kernel mode. This procedure must be followed when reading and writing from files, creating a new process, etc.
Application code never runs in kernel mode, only the system call itself which is kernel code. When the system call is complete, a return value is produced and the process returns to user mode with the inverse context switch.
There are other times when the system is in kernel mode that has nothing to do with processes, suchs as when handling hardware interrupts or running the scheduling routines and other management tasks for the system.
Daemons
Is a background process whose sole purpose is to provide some specific service to users of the system.
- Daemons can be efficient because they only operate when needed
- Many daemons are started at boot time
- Daemon names often (but not always) end with d
- Some examples include httpd and udevd
- Daemons may respond to external events (udevd) or elapsed time (crond)
- Daemons generally have no controlling terminal and no standard input/output devices
- Daemons sometimes provide better security control
When using SysVinit, scripts in the /etc/init.d directory start various system daemons. These scripts invoke commands as arguments to a shell function named daemon, defined in the /etc/init.d/functions file.
To see an example of how this is done look at the script for the httpd service in the /etc/init.d directory. Systems using systemd use similar methods for daemons.
Kernel Created Processes
Not all processes are created, or forked from user parents. The Linux kernel directly creates two kinds of processes on its own initiative. These are :
- Internal kernel processes
- Perform maintenance work, such as making sure buffers get flushed out to disk, that the load on different CPUs is balanced evenly, that device drivers handle work that has been queued up for them to do, etc. These processes often run as long as the system is running, sleeping except when they have something to do.
- External user processes
- These are processes which run in user space like normal applications but which the kernel started. There are very few of these and they are usually short lived.
To see the processes of this nature, run
$ ps -elf
They will have PPID=2 which refers to kthreadd, the internal kernel thread whose job is to create such processes and their names will be encapsulated in square brackets, such as [ksoftirqd/0]
Process Creating and Forking
When a given process creates a new child process this is called forking.
When computers had a single processor, the parent process stops while the child process runs, "Children comes first". However in modern systems with multi CPU, the trend is that both process keeps running simultaneously on different CPU´s.
When the parent dies and the child adopt its parent pid, this is called exec.
In order to imagine a multiprocess scenario consider a web server that creates a new process each time a new client requires a connection or it may create a new thread as part of the same process. In Linux there is not much difference on a technical level between creating a full process or just a new thread as each mechanism takes about the same time and uses roughly the same amount of resources.
Other sample is the sshd daemon which listen for ssh request from remote users. When a request arrives the sshd daemon start the authentication process, if it happens to be successful then the process will copy itself and assign the new process to the user whose is requesting the new connection.
Creating Processes in a Command Shell
What happens when a user executes a command in a command shell interpreter, such as bash ?
- A new process is created [forked from the user's login shell]
- A wait system call puts the parent shell process to sleep
- The command is loaded onto the child process's space via the exec system call
- The command completes executing, and the child process dies via the exit system call
- The parent shell is re-awakened by the death of the child process and proceeds to issue a new shell prompt. The parent shell then waits for the next command request from the user, at which time the cycle will be repeated
If a command is issued for background processing
- by adding an ampersand & at the end of the command line
The parent shell skips the wait request and is free to issue new shell prompt immediately allowing the background process to execute in parallel. Otherwise for foreground process the parent shell waits until the child process has competed or is stopped via a signal.
Some shell commands (such as echo and kill) are built into the shell itself and do not involve loading of program files. For these commands, no fork or exec are issued for the execution.
Using nice to Set Priorities
Process priority can be controlled through the nice and renice commands. Since the early days of UNIX the idea has been that a nice process lowers its priority to yield to others. Thus the higher the niceness is, the lower the priority.
Range of priority
- -20 "Highest priority"
- +19 "Lowest priority"
Samples
$ nice -n 5 command [ARGS]
Which would increase the niceness by 5. This is equivalent to doing
$ nice -5 command [ARGS]
If you do not give a nice value the default is to increase the niceness by 10. If you give no arguments at all, you report your current niceness.
$ nice 0 $ nice cat & [1] 24908 $ ps -l F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 0 S 1000 2644 2631 0 80 0 - 30173 wait pts/0 00:00:00 bash 0 T 1000 2748 2644 0 90 10 - 28083 signal pts/0 00:00:00 cat 0 R 1000 2767 2644 0 80 0 - 38533 - pts/0 00:00:00 ps
Modifying the Nice Value
By the dault only a superuser can decrease the niceness. However it is possible to give normal users the ability to decrease their niceness within a predetermined range, by editing
/etc/security/limits.conf
To change the niceness of an already running process we can use the renice command, for example
$ renice +3 13848
Which will increase the niceness by 3 of the process with pid=13848. More than one process can be done at the same time and there are some other options.
Static and Shared Libraries
Programs are built using libraries of code, developed for multiple purposes and used and reused in many context.
There are two types of libraries
- Static
- The code for the library functions is inserted in the program at compile time, and does not change thereafter even if the library is updated
- Shared
- The code for the library functions is loaded into the program at run time, and if the library is changed later, the running program runs with the new library modifications.
Using shared libraries is more efficient because they can be used by many applications at once, memory usage, executable size, and application load time are reduced.
Shared libraries are also called DLLs (Dynamic Link Library).
Shared Libraries Versions
These libraries needs to be fully versioned, the syntax is like
LibraryName.so.M.m.p
Where
- M
- Stands for Major
- m
- Stands for minor
- p
- Stands for patch
Finding Shared Libraries
A program which uses shared libraries has to be able to find them at run time.
ldd can be used to ascertain what shared libraries an executables requieres. It shows the soname of the library and what file it actually points to:
[abernal@localhost ~]$ ldd /usr/bin/vi
linux-vdso.so.1 (0x00007ffe3f8fd000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fc45bf60000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007fc45bd36000)
libacl.so.1 => /lib64/libacl.so.1 (0x00007fc45bb2c000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc45b76b000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fc45b4fb000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fc45b2f6000)
/lib64/ld-linux-x86-64.so.2 (0x000055e4e6457000)
libattr.so.1 => /lib64/libattr.so.1 (0x00007fc45b0f0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc45aed3000)
ldconfig
Is generally run at boot time (but can be run anytime), and uses the file /etc/ld.so.conf, which lists the directories that will be searched for the shared liraries. ldconfig must be run as root and shared libraries should only be stored in system directories when they are stable and useful.
Besides searching the data base built up by ldconfig, the linker will first search any directories specified in the environment variable LD_LIBRARY_PATH, a colon separated list of directories, as in the PATH variable. So you can do :
$ LD_LIBRARY_PATH=$HOME/foo/lib $ foo [args] $ LD_LIBRARY_PATH=$HOME/foo/lib foo [args]
Lab 20.1: Controlling Processes with ulimit
Please do:
$ help ulimit
and read /etc/security/limits.conf before doing the following steps.
1. Start a new shell by typing bash (or opening a new terminal) so that your changes are only effective in the new shell. View the current limit on the number of open files and explicitly view the hard and soft limits.
2. Set the limit to the hard limit value and verify if it worked.
3. Set the hard limit to 2048 and verify it worked.
4. Try to set the limit back to the previous value. Did it work?
Solution
1. Check the limits for the amount of open files
$ bash $ ulimit -n 1024 $ ulimit -S -n 1024 $ ulimit -H -n 4096
2. Set the hard limit
$ ulimit -n hard $ ulimit -n 4096
3. Set a limit of 2048
$ ulimit -n 2048 $ ulimit -n 2048
4. Try to change the limit again
$ ulimit -n 4096 bash: ulimit: open files: cannot modify limit: Operation not permitted $ ulimit -n 2048
You can’t do this anymore!
Note that if we had chosen a different limit, such as stack size (-s) we could raise back up again as the hard limit is unlimited
Lab 20.2: Examining System V IPC Activity
System V IPC is a rather old method of Inter Process Communication that dates back to the early days of UNIX.
It involves three mechanisms:
1. Shared Memory Segments
2. Semaphores
3. Message Queues
More modern programs tend to use POSIX IPC methods for all three of these mechanisms, but there are still plenty of System V IPC applications found in the wild. To get an overall summary of System V IPC activity on your system, do:
$ ipcs ------ Message Queues -------- key msqid owner perms used-bytes messages ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01114703 0 root 600 1000 6 0x00000000 98305 coop 600 4194304 2 dest 0x00000000 196610 coop 600 4194304 2 dest 0x00000000 23068675 coop 700 1138176 2 dest 0x00000000 23101444 coop 600 393216 2 dest 0x00000000 23134213 coop 600 524288 2 dest 0x00000000 24051718 coop 600 393216 2 dest 0x00000000 23756807 coop 600 524288 2 dest 0x00000000 24018952 coop 600 67108864 2 dest 0x00000000 23363593 coop 700 95408 2 dest 0x00000000 1441811 coop 600 2097152 2 dest ------ Semaphore Arrays -------- key semid owner perms nsems 0x00000000 98304 apache 600 1 0x00000000 131073 apache 600 1 0x00000000 163842 apache 600 1 0x00000000 196611 apache 600 1 0x00000000 229380 apache 600 1
Note almost all of the currently running shared memory segments have a key of 0 (also known as IPC_PRIVATE) which means they are only shared between processes in a parent/child relationship. Furthermore, all but one are marked for destruction when there are no further attachments. One can gain further information about the processes that have created the segments and last attached to them with:
$ ipcs -p ------ Message Queues PIDs -------- msqid owner lspid lrpid ------ Shared Memory Creator/Last-op PIDs -------- shmid owner cpid lpid 0 root 1023 1023 98305 coop 2265 18780 196610 coop 2138 18775 23068675 coop 989 1663 23101444 coop 989 1663 23134213 coop 989 1663 24051718 coop 20573 1663 23756807 coop 10735 1663 24018952 coop 17875 1663 23363593 coop 989 1663 1441811 coop 2048 20573
Thus, by doing:
$ ps aux |grep -e 20573 -e 2048 coop 2048 5.3 3.7 1922996 305660 ? Rl Oct27 77:07 /usr/bin/gnome-shell coop 20573 1.9 1.7 807944 141688 ? Sl 09:56 0:01 /usr/lib64/thunderbird/thunderbird coop 20710 0.0 0.0 112652 2312 pts/0 S+ 09:57 0:00 grep --color=auto -e 20573 -e 2048
we see thunderbird is using a shared memory segment created by gnome-shell. Perform these steps on your system and identify the various resources being used and by who. Are there any potential
leaks (shared resources no longer being used by any active processes) on the system? For example, doing:
$ ipcs .... ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status .... 0x00000000 622601 coop 600 2097152 2 dest 0x0000001a 13303818 coop 666 8196 0 ....
shows a shared memory segment with no attachments and not marked for destruction. Thus it might persist forever, leaking memory if no subsequent process attaches to it.