Change ownership of symlink – Linux

Problem – Question
I had to change the user and group of a symbolic link.
The usual chown -R username:group target_to_change/ did not work.

Answer
When auto-completing the target_to_change like I always do, you will notice the tab complete puts a slash on the end like this “target_to_change/”. You have to remove the trailing slash or the aforementioned command will not work. I guess it does not recognise the name of the symlink unless the name is given verbatim.

Tim

How to add a new user Ubuntu, Red Hat, Linux

How to add a new user Ubuntu, Red Hat, Linux
I know that you can read the man page by typing
man useradd
However I like jotting down a few notes via this blog to ensure that I am not trying to unravel the cryptic official documentation when I need to add another user in the future.

Home directory

-b or --base-dir
This is specifying the home directory of the SYSTEM eg “/home” not the USER eg “/home/theNewUser”
When only specifying -b or –base-dir
useradd theNewUser -b "/home"
The system will not create a home directory for the user such as “/home/theNewUser” if we want the system to create a home directory we need to add the -m argument at the end of the useradd command like this
useradd theNewUser -b "/home" -m
This will take the “/home” we provided and append “theNewUser” to it to create “/home/theNewUser” behind the scenes.

-d or –home
This is specifying the complete home directory of the USER eg “/home/theNewUser” not the SYSTEM home eg “/home”
If you give a path that is bogus like this
useradd theNewUser -d "blahblah/donkey" -m
You get an error like this
useradd: invalid home directory

When specifying -d or –home, the directory for the not yet created user must exist, If the home directory for the not yet created user does not exist it will NOT be created.
If the directory does not yet exist and we want the system to create a home directory behind the scenes we need to add the -m argument at the end of the useradd command like this
useradd theNewUser -d "/home/theNewUser" -m

Password
I set the password after creating the new user using the passwd command like this
sudo passwd theNewUser
I believe that the -p argument is not a good idea given what I understand from the man page
-p, –password PASSWORD
The encrypted password, as returned by crypt(3). The default is to disable the password.
Note: This option is not recommended because the password (or encrypted password) will be visible by users listing the processes.

Shell
-s or –shell like this
sudo useradd aaaa -s "/bin/bash" -d "/home/aaaa" -m

Group
sudo useradd aagg -g aaag -s "/bin/bash" -d "/home/aagg" -m
Note when using lowercase g (-g) you are setting the primary group of the user, whatever you add after lowecase g will overwrite the one and only primary group assigned to that user. when using the upper case G, you are able to add additional groups to which the user can belong. This is handy when doing usermod -G secondGroup theNewUser

Need to convert mp4 to mp3 and then split audio into smaller chunks

Literally have less than 10 minutes this morning to do a quick job for my father. Task is as follows:

  • Download video with information he wants
  • Convert to mp3 so he can play it in his car
  • Split the mp3 into smaller chunks because the mp3 player he has does not keep its place when he turns the car off (has to start at the beginning of hour long audio file every time he gets back in the car to drive)

Thank god for Ubuntu… here is what I did

apt-get install ffmpeg libavcodec-extra-52

Converts file from mp4 to mp3

ffmpeg -i a_file.mp4 -vn -acodec libmp3lame -ac 2 -ab 160k -ar 48000 b_file.mp3
apt-get install mp3splt

Splits audio into 5 minute blocks

mp3splt -f -t 5.0 -a -d split b_file.mp3



Shredding devices (erasing data) using dd command (Ubuntu Linux)

Instead of using the shred command that comes with Ubuntu Linux, I decided to simply use the dd command and shove the output of /dev/zero into the device that I want to shred /dev/zero will basically just keep supplying zero’s untill they are not requested anymore. This procedure will fill the device with zeros or null characters.

Please be carefull using this command as it could completely erase important files (pay attention and double check device names etc)

Command:

sudo dd if=/dev/zero of=/dev/the_device_to_be_nuked bs=4k conv=notrunc

Output:

22964740+0 records in
22964740+0 records out
94063575040 bytes (94 GB) copied, 2771.83 s, 33.9 MB/s

PLEASE NOTE: If the size of GB copied does not equal the size of the device (as I experienced running the above command) then the zeroing has not been successful. You should get the following message “No space left on device” if the zeroing has been successful. This means that it has filled the entire device. I had to run the command again.

Once finished I want to see if there is any data that has not been zeroed out. This command basically reads the entire device in a linear fashion converting characters to Hex and reports on any characters that are not (^) zero’s 0. By the way this command takes quite a long time, nearly as long as the command above however it is worth doing in order to make sure that the device is really zeroed out.

Command:

dd if=/dev/the_device_to_nuke | hexdump -C | grep [^00]

Update: 2011-06-19

The output from the dd command that checks for anything that is not 00 was ” |…………….| * eeffa000″ so I am guessing that everything was not zeroed out. I may be way off the mark here but I wrote a script to replace everything that is not a zero with a zero. The script is as follows (warning this is just an experiment and I am learning, please do not take this as gospel… if it does not work for you I would be glad to hear any suggestions)

#!/bin/bash
data=`dd if=/dev/drive_to_be_nuked | hexdump -C | grep [^00]`
echo "********"
echo $data
echo "********"
foonum=${#data[@]}
echo "foonum is $foonum"
for ((i=0;i<$foonum;i++)); do
    content=${data[${i}]}
    `dd if=/dev/drive_to_be_nuked | sed 's/$content/00/g' | dd of=/dev/drive_to_be_nuked`
done

What is the maximum file length + Unix + Linux

I decided to find out by whipping up a quick bash script.

#!/bin/bash

function make_dir {
echo "creating file with " $size " characters"
touch $dir_name
if [ $? -ne 0 ]
then
 exit 1
fi
old=$dir_name
add="a"
dir_name=$dir_name$add
size=`expr $size + 1`
rm -rf $old
}

dir_name="a"
size=1
while [ $? -eq 0 ]
do
 make_dir
done

I got the following output

...creating file with  234  characters
creating file with  235  characters
creating file with  236  characters
creating file with  237  characters
creating file with  238  characters
creating file with  239  characters
creating file with  240  characters
creating file with  241  characters
creating file with  242  characters
creating file with  243  characters
creating file with  244  characters
creating file with  245  characters
creating file with  246  characters
creating file with  247  characters
creating file with  248  characters
creating file with  249  characters
creating file with  250  characters
creating file with  251  characters
creating file with  252  characters
creating file with  253  characters
creating file with  254  characters
creating file with  255  characters
creating file with  256  characters

So there ya go. Give it a try on some other platforms I would love to see if there are differences.

Backing up a DVD fast, achieving high quality, using the Ubuntu command line.

Update:

See the following link for an update “burning a dvd using shell script” (this is an automated script)

Burn a DVD using Shell Script

Burn DVD

Firstly to get the data from the disk to the hard drive, insert DVD into drive and issue the following command. If you are unsure of the device simply type df and you will see it listed (look for the name of the DVD to identify the correct line)

dvdbackup --verbose --mirror --input=/dev/sr0 --name=movies_name --output=/home/user/where/to/store/the/movies

Convert VOB files to high quality avi files

ffmpeg

ffmpeg -i VTS_02_0.VOB -s hd1080 -sameq 1.avi

ffmpeg -i VTS_02_1.VOB -s hd1080 -sameq 2.avi

etc etc

Note: There are many options for ffmpeg, please see “man ffmpeg” command for more details.

Join/combine/merge the avi files by installing transcode and transcode-utils

sudo apt-get install transcode transcode-utils

Join command

avimerge -o master.avi -i 1.avi 2.avi

The file called master.avi will now contain 1.avi and 2.avi in that order

iFolder on Ubuntu

Packages

sudo apt-get install build-essential automake autoconf mono-complete liblog4net1.2-cil uuid-dev libxml2-dev mono-apache-server mono-apache-server2
gsoap libapache2-mod-mono libncurses-dev libtool g++-4.3 gcc-4.3 apache2

Modules to enable

sudo a2enmod ssl
sudo a2enmod rewrite

See /usr/share/doc/apache2.2-common/README.Debian.gz for ssl information

Set hostname

First set your hostname using the hostname command, then edit the hosts file using tab seperation (all entries per host on one line)

hostname servershostnamegoes here
sudo vim /etc/hosts
127.0.0.1    servernamegoeshere    localhost

Install Flaim

cd ~
mkdir tmp
cd tmp
wget http://forge.novell.com/modules/xfcontent/private.php/flaim/development/flaim/downloads/source/libflaim-4.9.989.tar.gz -O - | tar -xzf -
cd libflaim-4.9.989/
sed -i "s/\-Werror//" Makefile
make
sudo make install

Install iFolder

cd ~/tmp
wget http://sourceforge.net/projects/ifolder/files/iFolder%20Clients/3.8.0/simias-1.8.3.9328.1.tar.gz/download -O - | tar -xzf -
cd simias-1.8.3.9328.1/
export CSC_LIBFLAG="/target:library"
export WSDL=wsdl2
export CSC2="gmcs -d:MONO -d:MONONATIVE"
export CSC="gmcs -d:MONO -d:MONONATIVE"
export MONO=mono
export CC=gcc-4.3
export CXX=g++-4.3

*NOTE - The following 3 lines are all one command*
sed -i -e "s+\$(GSOAP_PATH)/wsdl2h+wsdl2h+g" -e "s+\$(GSOAP_PATH)/soapcpp2+soapcpp2+g"
-e "s+\$(GSOAP_PATH)/stdsoap2.c+/usr/include/gsoap/stdsoap2.c+g"
-e "s+\$(GSOAP_PATH)/stdsoap2.h+/usr/include/stdsoap2.h+g" src/core/libsimias/Makefile.am

make
sudo make install
sudo updatedb
locate simias-server-setup
sudo /where/it/is/simias-server-setup

The only changes I made during the set up prompts was the choice not to use LDAP and the insertion of my hostname (which we set in the first few steps of this doco), pretty much kept everything else as the default.

You may want to open port 443 on the firewall (below is an example of just opening 443 to my internal network).

iptables -A INPUT -s 192.168.0.0/24 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT

sudo sed -i 's/apache2\/mod_mono.conf/apache2\/mods-enabled\/mod_mono.conf/g' /etc/apache2/conf.d/simias.conf

sudo mkdir -p /var/www/.config/.mono/
sudo chown -R www-data:www-data /var/www/.config/

Locate the ifolder-admin-setup script and run it, make sure that you apply the correct apache user and group eg www-data

sudo /where/it/is/ifolder-admin-setup

Locate ifolder-web-setup (same as above and run it as sudo be sure to enter your external URL instead of accepting …localhost:443 and again make sure that the corrent apache user and group are used.

sudo /where/it/is/ifolder-web-setup

fix up the conf files

sudo sed -i 's/apache2\/mod_mono.conf/apache2\/mods-enabled\/mod_mono.conf/g' /etc/apache2/conf.d/ifolder_admin.conf

and

sudo sed -i 's/apache2\/mod_mono.conf/apache2\/mods-enabled\/mod_mono.conf/g' /etc/apache2/conf.d/ifolder_web.conf 

then restart apache2

/etc/init.d/apache2 restart

Gochyas (Read these first)

SSL HostName Issue

You will have to create a self signed Cert as part of this install if you are choosing SSL when installing iFolder. You will have to Google that but take note of the following during the creation of the cert.

Did not get much joy in the browser so I checked the /var/log/apache2/error.log and saw the following

[warn] RSA server certificate CommonName (CN) `ubuntu’ does NOT match server name!?

To solve this make sure that you have your hostname set as the ServerName in the apache config files that are being used. And most importantly this name must patch the setting called Common Name (eg, YOUR name) []: that you have to enter while creating your certificate eg running

sudo openssl req -new -x509 -days 365 -nodes -out /etc/ssl/certs/server.cert -keyout /etc/ssl/certs/server.key

mod_mono

During the install of mod_mono I chose yes for the following (the default is no)

Configuration file `/etc/apache2/mods-available/mod_mono.conf'
 ==> File on system created by you or by a script.
 ==> File also in package provided by package maintainer.
   What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : background this process to examine the situation
 The default action is to keep your current version.
*** mod_mono.conf (Y/I/N/O/D/Z) [default=N] ? Y

Managing processes on Ubuntu

The issue

I want to be able to have all the critical services start up in the event that the server is rebooted.

Init scripts

The first step is to write the init scripts, name it (we will call ours app) save it to the /etc/init.d/ directory and make it executable by root user. Below is an example of a really simple one that could do with allot more tweaks, but lets just get the concept first.

#!/bin/sh
case $1 in
        start)
                /home/user/folder/start_app.sh
        ;;
        stop)
                /home/user/folder/stop_app.sh
        ;;
        restart)
                /home/user/folder/restart_app.sh
        ;;
        *)
                echo "Useage sudo /etc/init.d/app2 {start|stop|restart}"
esac

Once you have create the init script run the following command to register your scripts for execution as your system changes run levels (single user, multi user, reboot etc).

sudo update-rc.d app defaults

How to back-up Zotero

This is a quick blog to show you how I am backing up my Zotero (Ubuntu and Firefox)

This is a script that I whipped up for myself, this runs every month using a cron job. It basically stops Firefox if it is running (freeing up the Zotero sqlite file) compresses the entire Zotero dorectory and then emails that to you.

#!/bin/bash
zotero_directory="/CHANGE/ME
tar_name="/CHANGE/ME/zotero.tar.bz2"
email="CHANGE@ME.COM"
regex=.*[Ff]irefox.*
echo "Checking Firefox processes"
echo "ps ax | grep -e $regex"
echo "Killing Firefox processes"
killall -r $regex
if [ $? == 1 ]
then
        echo "Firefox was not in use anyway"
else
        echo "Firefox process killed"
fi
echo "Compressing Zotero Directory $zotero_directory to Tar File $tar_name"
if [ -f $tar_name ]
then
        echo "Tar file already exists, script must not have completed last time, removing tar file now"
        rm -rf $tar_name
fi
echo "Creating new tar file"
tar -cjvf $tar_name $zotero_directory
echo "Sending Tar File $tar_name to $email"
mutt -s "cairss zotero backup" -a $tar_name $email
sleep 10
rm -rf $tar_name

Sudoers using Ubuntu

The following blog post is a running account of how I am setting up privileges on some new machines. The intention of this post is to serve as a quick reference for myself instead of having to read the really long man pages every time I want to make a small change.

Adding an admin group

This step creates a new group that will have administration members only allowed. Must run this command as root of course.

groupadd administrators

Modifying the user to be part of the new group

This step adds the very trusted and privileged user (let’s say in this case bob) to be a member of the administrators group. Again run this as root.

usermod append -G administrators bob

Editing the sudoers file to allow root access to the administrators group

visudo

Then add the following lines

# This is a comment that says the group administrators has full root privileges
%administrators ALL=(ALL) ALL

This is a very simple example but is enough to get started more on this later