Feb 19
Linux
Training Session 01
http://mreschke.com/mrcore/mrticles/Linux%20Training
Material Covered
What is the Base System
Directory Layout (What and Where everything is)
Everything is a File
First, I would suggest that sometime you google and read about the history of linux and what exactly linux is. No need to read a huge article, just find a nutshell explanation. For those of you who like to evangelize about linux, it's always good to get your facts straight. Also learn what exactly Mac OSX, GNU, and BSD are too. You'll find they are all much the same (unix like, posix compliant groups of software). Along your history study, you will find important things like "linux is NOT an operating system" (it simply refers only the the kernel), but Linux/GNU is an OS. You will learn what GNU is, how GNU apps are different from BSD apps, where BSD came from (unix converted to x86 PC platform), and all about the GPL licensing that keep linux free (no one will ever own linux).
A Brief History of Linux:
http://www.thestandard.com/article/0,1902,9457,00.html
A Brief History of Mac OSX:
http://osxbook.com/book/bonus/ancient/whatismacosx/history.html
A Brief History of GNU:
http://www.gnu.org/gnu/gnu-history.html
A Brief History of BSD:
http://www.freebsd.org/doc/en/books/handbook/history.html
I don't really know where to start with a linux training session. I don't really know of an easy way to organize which lesson comes first either, so I'm just going to dive in somewhere and by the end of all these trainings (hopefully) you will know a lot of insanely nerdy info about linux and computers/networking/administration/programming/databases in general. Im no teacher. Im just creating some nice articles for my websites wiki, live on google wave.
Linux Filesystem Layout
This seems to be a hard part for a lot of people, and I was no exception. There are a lot of folders and files with strange, short names. When I started out with linux I never spent the time to learn exactly where everything is, it took about 6 months of just using linux to naturally understand where most things were, and I still benifited greatly from reading more details about it.
Terminology or Lingo
Real quick, lets get a bit of lingo down. The root filesystem refers to the highest (some say lowest) or first point in a filesystem hierarchy. The parent of all the other filesystems mounted in a Unix filesystem tree. It is the base of which all other files and folders reside. Please note that this is NOT the same as the root user. The root user refers to the absolute master admin of the system, not the file structure. So "becoming root" or "login as root" is NOT the same as "goto root" or "open the root slash home slash user1 folder", ya dig?
Everything in linux is a file
Or better yet, "everything is linux is a file is represented as a file or a process". In windows, the filesystem only holds what most think of as true files, chunks of data like text files, music, executable programs, movies... But in linux, everything is a file, like your keyboard, mouse, sound card, video card, hard drive, hard drive partitions etc...
Every piece of hardware on your system has an associated (represented) file with it. And this might get confusing, but every thing that is "going on" with your computer right now (all the kernel is doing, all that is in your ram memory for every program) has a file that contains that "virtual" content. Perhaps I will go into that in more detail later, but just know that the linux filesystem that you see and use does not just contain straight forward files and folders. It has devices files, virtual system files and kernel files, process files and mount files.
This simple idea is what helps make linux incredibly powerfull as you will see through these lessons. Because everything is a file, you can access it with ease. You can manipulate your hardware (or see whats its doing) with a simple command or script, you can control your kernel by changing "virtual" files, you can gather any type of hardware information by reading a "virtual" file. Its truely amazing, unfortunately you can only truely see its benifits as you continue to use linux in a more advanced manner (which we'll get too sometime).
So, this is why the filesystem can seem quite confusing at first. But if you simply ignore (for now) all the "fake" files and focus on the real files, you will find that it's quite easy to understand.
So real quick, here is a basic list of folders on the linux root, and you can see your root by opening a terminal (or bash shell, or command prompt, all the same) window and typing "ls -al /"
bin
boot
dev
etc
home
lib
lib32
lost+found
media
mnt
opt
proc
root
sbin
selinux
srv
sys
tmp
usr
var
So there are about 20 folders on a standard linux OS. Sometimes you might have a few files scattered on the root too, you can easily tell if what your looking at is a folder by looking at the first part in each line
example:
drwx-xr-x 20 root root 4096 Feb 9 12:58 bin
Notice the first character is a d, that means its a directory, if the first character is just a -, then its a file, if the first character is an l (thats lowercase ell), then it's a symbolic link (kinda like a windows shortcut or a mac alias, but much better, well get in to links sometime).
So out of all those root folders, these contains all the real files? Yes, eveything on your system, real or fake is in those folders.
Out of all those folders, these 3 contain "virtual" items
dev
proc
sys
Let me clarify that even though dev, proc and sys "contain" virtual files/folders, the actual folder itself is very real, meaning there is an actual folder on your hard drive, it's just empty as far as the hard drive is concerned. All the files you see when you run "ls -al /dev" or "ls -al /proc" are virtual (they are devices or files in memory used by the kernel or other processes).
Base System
Let me explain what the "base system" is before I explain the directory layout, because a lot of folders contain "base" programs
The base system refers to the 'core' or 'most basic' set of files used by any linux system. Basic files like 'cp' to copy files, or 'rm' to delete a file... There are several thousand executable files (programs) and config files in the 'base' system. Without this small set of files, a basic linux system would not be complete. Imagine if you had a linux kernel running, but no shell, you wouldn't be able to interact with the computer and it would be useless. The shell is a file, if you are using bash, that file is /bin/bash, it is of course included in the 'base' system. This 'base' does not include any graphical interface (GUI) or any extra tools (like ripping cds or converting movies). Most of the 'base' executable files are located in /bin and /sbin folders. Also note, that the "base system" is NOT a standard, any linux distro can choose what files go into their base system. But most base system are nearly the same, the will always have the most basic tools for you to actually 'use' the system.
File Lingo
A "binary executable" is simply a program, something that you run or execute. You cannot read the file like you can a text file, it's not human readable because it's been compiled into machine code to be executed by your system. Binary files basically means "compiled files".
A "binary library" is code that has been compiled, but is NOT executable. It is compiled code that is used by other programs, a collection of compiled code. Just like a .dll file in windows, they are binary libraries.
A "config file" is a standard text (human readable) file that is used to configure some or several programs. Most programs in linux store their variable configuration data in a file so that the program can be tweeked to fit the users needs. Since they are just plain text files, they can be easily read and edited in a simple text editor.
A "script" is a standard text file that can be executed. Kind of like a "binary executable" in that it can be executed, but NOT like one because its NOT compiled binary, its just plain text. When you execute this script, it is parsed by a programming language (like perl, bash, php...) which compiles it into machine code (binary) and executes it (does this behind the scenes, automatically).
A " text file" or "plain text" does not just mean a text (.txt) document. It means anything that is human readable (not compiled), like config files, or text documents, scripts, html, etc... Most files in linux are text files because linux mostly runs on bash scripts and config files. This is what makes it incredible to tweek and play with.
A "virtual file" is a fake file, meaning its not real data on your hard drive, it's just data in ram (but looks like a file) that will be destroyed when you reboot. These files are used for the kernel and tons of other things. These files are usually just plain text files that you can read in a text editor. Great for looking at the inner workings of a system!
A "device file" is also fake, meaning not real data on your hard drive. Its a file that "represents" a piece of hardware on your computer, like a sound card, hard drive, video card, etc...
Folder Explications
Ok, now to explain what each folder contains.
Nearly everything in Linux is governed by standards, its what keeps if from spinning out of control. You can see the actual filesystem standards here http://www.pathname.com/fhs/pub/fhs-2.3.html
For a quick simple explication http://www.nixtutor.com/linux/understanding-the-linux-directory-layout/ or google around
bin
contains all the 'base' executable files that can be executed and used by everyong on the system. Executables in this directory usually will never change (meaning nothing added, nothing taken away), it is populated when you install your system. So when you install a separate piece of software, it does not go here. This is "base system" executables only. Have a look at yours by typing "ls -al /bin". Remember, it contains ONLY executables, no config files or libraries... You will notice programs in there that you use everyday, like 'rm', 'cp', 'mkdir', 'ping'... etc. bin is short for 'binary', so think "binary executables"
boot
Contains only the files that are necessary to get a system up and running. It has your kernel, which is the file usually named 'vmlinux...' something. Like my arch linux is 'vmlinuz26' and my ubuntu is 'vmlinuz-2.6.27-7-generic' Those numbers mean the kernel version. The boot folder also contains an "initramfs" file (short for Initial Ram File System). This initramfs is usually called kernel26.img or initrd.img-2.6.27-7-generic, etc... I know it's strange that its called kernel26.img, even though its NOT the kernel. It is the kernels first (initial) filesystem. The kernel has to get its drivers for your monitor, hard drives...somewhere right? It gets them here and then loads your normal filesystem (the one on your hard drive). It'sa very compilcated boot process that I will probably cover at a future time. You can google initramfs or initrd (these two are NOT the same, but are used like they were the same). This folder also contains all your boot loader files, most linux distros use a bootloader program called GRUB. When you get into multibooting, you can edit your /boot/grub/menu.lst file to list out all your OS at boot!
dev
Contains all the virtual "device files" I mentioned earlier. All your hardware devices are represented by files stored in this directory. To keep things simple, just ignore this directory for now. dev is short for 'device'. This folder ONLY contains device files.
etc
Contains almost every "config file" for every program on your system. Its one of the great things about linux, most configs are stored right here. Note, this folder ONLY contains config files.
home
Contains subfolders for each user on your linux system (except the root user, whos home folder is /root). So for example, your name is john, so all your personal settings (like GUI changes, backgrounds, bash profiles, appearance settings..) would be in /home/john/ This is where you would also keep all your personal data, movies, pictures, your own scripts... In windows, this would be like 'c:\Documents and Settings\'
lib
Contains system "binary files". Or code used by other programs on your system. In windows, this would be like c:\windows\system32 which contains all your windows .dll files. In linux libraries are like windows .dll files (which are also called libraries, or dll libraries). lib is short for 'library'
lib32
Same as /lib except for 32bit libraries. If you are running a 64bit version of linux, the /lib folder will be all 64bit libraries and the /lib32 folder will be 32bit ones.
lost+found
Your hard drives filesystem is probably ext3 or ext4. This filesystem uses a journal to help recover quickly from system crashes or power outages. If your system crashes, a check disk will start on next boot. If the disk check finds any corrupt files it will recover them and place them in /lost+found for you to fix and move later.
media
I use this to mount all sorts of crap, but it is meant to be used only as a mount place for removable media, like CD-ROMS, floppies, USB sticks. If you plug in a USB flash drive your system will automatically create a folder in /media and mount the USB memory stick there.
mnt
I use this for all sorts of mount too, but its made for the administrator (that would be YOU) as a place to temporarily mount various filesystems as needed. By default this dir is empty. If you throw in another hard drive and want to access it, just make a new dir like "sudo mkdir /mnt/newdisk" and then mount that drives partition to that folder "sudo mount /dev/sdaX /mnt/newdisk". mnt is short for mount.
opt
This is used by some applications that you install. Most apps will go in /usr, but sometimes the developer makes them go here. It's not used very much. This would KIND OF be like c:\Program Files in windows. Again most software goes to /usr.
proc
This is a virtual directory, much like /dev except that it contains process information. A process is something running on your computer, usually in the background. On your system run 'ps -aux' and you will see all the running processes on your computer. These processes store them information in ram, which are also files right? These virtual process files are stored here in /proc. Open it up and check it out. For example you can see all about your memory usage by typing 'cat /proc/meminfo' or if you want to know what sound card you have, run 'cat /proc/asound/cards'. proc is short for process. Again, these are virtual, ever changing files.
root
This is like the /home/john/ directory, except for the root user. Usually just contains the root users bash profile and history files.
sbin
OK, the /sbin folder is just like /bin except that it contains executables that should ONLY be run by root (the admin). Things like fdisk (to partition hard drive, udev to control hardware... ONLY contains executables and just like /bin, this is part of the 'base system' Usually nothing changes in here, nothing is added, nothing is deleted, its populated when you install your linux OS. sbin is short for System Binary
selinux
This is used by a program called 'selinux', its an application level linux security tool. Most distros do not have this, but some do.
srv
This is empty by default (maybe has a few folders, but they are usually empty too). If you were to run a web server or ftp server, or any kind if fileserver, you would store the builk data and code here. Example, all the .php files used to run my wiki site are stored on my server in /srv/http/mreschke/ srv is short for server
sys
Another virtual directory, very similar to /proc but used by plug and play devices. sys is short for system
tmp
just a temporary folder, though you should not delete its contents because some running programs could be using it, and you might cause them to crash. On some system this is deleted on every reboot, so don't put anything important in here. I use this for quick mounting and playing. Its readable and writable by ALL users. tmp is short for temporary.
usr
OK, the /usr folder is the builk of your software. After a fresh "minimal base system" it's quite empty, but this is where most programs get installed too. Not just programs though, also the files used by those programs (except for configs which go in /etc). usr stands for user, think if it like user (meaning YOU) installed programs, not 'base system' programs. If you look in the /usr folder is split out into other similar looking folders, like
bin (for binary executables)
include (code files used by the programs)
lib (just like /lib except for the installed programs in /usr, where as /lib is used for 'base system' programs AND /usr programs)
local (originally ALL user installed programs were supposed to go here in /usr/local, but no one seems to catch on. If you look in this /usr/local folder it looks almost identical to /usr, it has a bin,include,lib,sbin,share.. folders. So is kinda a crap shoot, some developers will build their software to install to /usr and some will choose to install to /usr/local. You can think if it like I am the "local" user, and these are my programs, NOTE: none of the 'base system' executables or files will ever go in /usr or /usr/local, they are always in /bin or /sbin. /usr and /usr/local for for program YOU install.
sbin (like /usr/bin except for system executables)
src (contains source code for some programs (if you choose to install the source). If you install the kernel source code, it will go here)
var
Contains information that changes a lot, like log files, backup files, your email, system caches, database files, etc... This folder is usually quite large as well, but not usually as large as /usr. If you want to view your system log, type "cat /var/log/message".
Holy crap! So really if you think about it, we have a lot of folders called bin, like /bin, /sbin, /usr/bin, /usr/sbin, /usr/local/bin, /usr/local/sbin. But by knowing the all bin (or sbin) folders simply and ONLY contain executable binaries it makes things a lot easier to understand. And by simply knowing the /bin and /sbin are 'base system' stuff which doesn't change, and the ones in /usr are the program YOU have installed helps too. Like if you were looking for a program you installed called leafpad, you know you can omit looking in /bin or /sbin because thats 'base' stuff.
Same with /etc. If you want to tweek something or edit a config file, it's in /etc
And folders that can easily be ignored for now are lib, include, src, sys, proc, dev, boot, lost+found... So really when you narrow it down it gets easier to remember and understand.
Next week (or whenever) I will go into commands, and we will see where things are much better. And so you start learning about various tasks, you will start using all these folders over and over and will become very familiar with them. But I still go back to a chart to refresh where something might be.
Feb 18
Emilio:
Just wanted to say thank so much for doing this its helping alot
Feb 19
Me:
No problem. This will be a great addition to my wiki. It will be nice when we get a few of these articles down. Then I can have a place to redirect people too who want a consolidated linux tutorial.
Feb 18
Me:
How did kde install on your mac?
Feb 18
Emilio:
ok it was pretty easy i just used the graphical install off the live cd
Feb 18
Emilio:
though i didnt want to screw around with the partitions too much. I had 2 others besides the osx partition installed ubuntu on one and i have a really huge swap space on the other
Feb 18
Emilio:
I need to find the right wireless driver though Im on it now wired. on the ubuntu site my model doesnt have official support for 9.10
Feb 18
Me:
maybe an older one will work? Google fwcutter too, maybe that will work, but probably not with the mac hardware
Feb 18
Emilio:
Wel there is a Broadcom driver that is used in 9.04 for my model I downloaded
Feb 18
Me:
oh, its broadcom? nice, in that case fwcutter will work I am sure. Try sudo apt-get install b43-fwcutter that might work
Feb 18
Emilio:
it worked
Feb 18
Me:
really, nice! IT's a good thing macs uses broadcom, it's the easiest to get going in linux
Feb 18
Emilio:
whats funny is I am booted into kubuntu and can't remeber what my wireless password is. its saved on the mac side.
Feb 18
Me:
FYI, sudo means run this command as root (admin) user. And remember I said linux uses packages? All the programs you'll ever want are at your finger tips. just type 'sudo apt-cache search *' to see most of the programs. To search do 'sudo apt-cache search firefox' or something, then to install 'sudo apt-get install package-name'
Feb 18
Me:
Check out some of my arch linux KDE shots
|
|
|
file |
|
|
|
file |
You'll like KDE i think
Feb 18
Emilio:
yeah it looks pretty sweet
Feb 18
Emilio:
So is the what i called the Broadcom driver really the "File" for the wireless card?
Feb 18
Me:
No the driver is actually a kernel module. In linux, kernel modules are the same as "drivers". But you can see your wireless card (and access it) like a file, it would be somewhere in /dev (not sure where). Really the driver itself, or the module, is just a chunk of code (file) added to the kernel. But you would not say "the file", just the "driver".
Feb 18
Emilio:
ah ok, that makes sense
Feb 18
John:
Those are pretty sweet pics!!!
Feb 18
Me:
heck yeah
Feb 18
Me:
wave getting a bit buggy
Feb 18
John:
I had to refresh This is all supper sweet info!!! How you holdin up captn?
Feb 18
Me:
Almost done, how YOU hold'n up man
Feb 18
John:
Im good gonna be looking a this doc tomorrow as well i hope it ok but i added a couple of lines at the top (brief Histotrys)
Feb 18
Me:
yeah, thats what I want, help colab and make it better, thanks!
Feb 18
John:
Can i post ?s in the text body?
Feb 18
Me:
yes, post anywhere
Feb 18
Emilio:
yeah it did a couple odd things to me too
Feb 18
John:
running slow on my machine but not to bad
Feb 18
Me:
close your browser and login again perhaps.
Feb 18
John:
Nah its just my lack of ram
Feb 19
Me:
Yeah so this wave is incredibly unstable. Perhaps it's just my firefox, it crashed about 12 times tonight. Bummer. Do any of you know of a better online collaboration tool? Sometime I would like to get a live desktop feed too. So I could stream my desktop to a web page so everyone could see what Im doing.
I read someone dogging on gwave because others were doing the same thing ages ago, but I cannot remember who.
Feb 19
Emilio:
not that I can think off the top of my head. Im sure there are some out there though
Feb 19
Me:
Feel free to keep pruning this wave too, adding links, pics, whatever. Edit all you guys want
Feb 19
Emilio:
oh Im sure I will. sooner or later Im gonna start using the os x unix and will post any differences i see. but it looks almost the same.
Feb 19
Me:
get yourself an avatar johnny