Linux


I cant take any credit for this script, I took the orginal from the solar.js.cx page and changed it to meet my requirements.
Here is my version

#!/usr/bin/perl
############################################
## This software is licensed under the LGPL.
## Visit http://solar.js.cx for more info.
############################################
use Device::SerialPort;
use LWP::UserAgent;
use Time::Local;
use HTTP::Status qw(:constants :is status_message);
$debug = 0;
$submit = 1;
$serial_port = "/dev/ttyr00";
$pvoutput_api ="<apikeyhere>";
$pvoutport_systemid = "<systemkeyhere>";

$serial_lock = "/tmp/ttyr00.lock";

## Wait until unlocked
while (-e $serial_lock)
{
  sleep (1);
}
$serial_port = new Device::SerialPort ($serial_port, "", $serial_lock);

$serial_port->baudrate(9600)				|| die "failed setting baudrate";
$serial_port->parity("none")				|| die "failed setting parity";
$serial_port->databits(8)				|| die "failed setting databits";
$serial_port->handshake("none")				|| die "failed setting handshake";
$serial_port->write_settings				|| die "no settings";

## Might need to tweak this if data is truncated.
$serial_port->read_const_time(40);

$serial_port->write("INV?\r");
($count, $xantrex_status) = $serial_port->read(255);
$serial_port->write("KWHTODAY?\r");
($count, $xantrex_kwhtoday) = $serial_port->read(255);
$serial_port->write("TIME?\r");
($count, $xantrex_time) = $serial_port->read(255);
$serial_port->write("POUT?\r");
($count, $xantrex_pout) = $serial_port->read(255);

$serial_port->close || warn "close failed";

$xantrex_wtoday =  $xantrex_kwhtoday * 1000;
#$time = timelocal($sec,$min,$hour,$mday,$mon,$year);
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
$mon += 1;
$date = sprintf "%04d%02d%02d", $year,$mon,$mday;
$ptime = sprintf "%02d:%02d", $hour, $min;

if ($debug == 1) {
	print "Current date/time is: $date - $ptime\n";
	print "Current inverter status is: $xantrex_status\n";
	print "Total KWH today is: $xantrex_kwhtoday\n";
	print "Total WH today is: $xantrex_wtoday\n";
	print "Current output is: $xantrex_pout\n";
}
chomp($xantrex_status);
my $length = length($xantrex_status);
$length -=1;
$status = substr $xantrex_status, 0, $length;
if (($submit == 1)  && ($status eq "ON")) {
	if ($debug == 1) {
		print  "Submitting data\n";
	}
	my $ua = new LWP::UserAgent;
	$ua->default_header( 'X-Pvoutput-Apikey' => $pvoutput_api, 'X-Pvoutput-SystemId' => $pvoutport_systemid );


	my $response 
	= $ua->post('http://pvoutput.org/service/r2/addstatus.jsp', 
	{ 'v2' => $xantrex_pout, 
	 'v1' => $xantrex_wtoday,
	 'd' => $date,
	 't' => $ptime
	  
	});
	my $content = $response->content;
	if ($debug == 1) {
		print "$content\n";
	}
	if ($response->is_error) {
		print "Error updating pvoutput.org $content\r\n";
	}
}

At $WORK2$ we had the express need to migrate from Asterisk 1.6 to 1.8. We are running Debian squeeze which only provides 1.6.9 as its stable version,which is fine, but when you have a bug that requires vendor support you get forced to upgrade. I didnt really want to rebuild packages from source (i hate building from source on a package managed system).

I discovered that asterisk provides its own debian/ubuntu repository for its packages which is awesome, so on our test box we quickly deployed it. And then during testing we ran into a snag. Namely that its next to impossible to find instructions on how to get the format_mp3.so module for mp3 support. In Debian its packaged with it, but due to some licensing who-ah asterisk doesnt ship them. i tried copying an older copy, and one out of the Debian squeeze repository but it wouldn’t load cause it was built with different compile time options.

After much reading and playing arround, i worked out how to build that module.. ANd since it took me the better part of a night to work it out, heres how i did it 🙂

Grab everything required to build the source for asterisk, some other libraries, the build tools, the actual source, and then configure it for our system.

apt-get install build-essential
apt-get build-dep asterisk
apt-get install libncurses5-dev libncursesw5-dev libxml2-dev
cd /usr/src
apt-get source asterisk
cd asterisk-1.8.11.1
./configure
make menuselect

In the make menu select, go into addons, and tick off format_mp3 (i think x was save and exit)

contrib/scripts/get_mp3_source.sh

This goes and grabs out of SVN the latest mp3 stuff, this is done for the packaging stuff as mentioned above

make addons

Assuming your make addons ran without errors, its built the module.. to install it run:
make addons-install
Now to restart asterisk
/etc/init.d/asterisk restart
and finally to confirm it actually loaded
asterisk -rx 'core show file formats'

as long as mp3 is in your list, your golden!

I used to have an encrypted volume on my file server, for storing sensitive things, but after an upgrade to the newest version of Debian, that stopped working. So after some mucking around and getting the old module for cryptoloop put back into my current kernel, I figured i might as well migrate over to the newer version, which uses cryptsetup, losetup, and luks.

Ive shamelessly stolen these commands (although ive changed some of them)from elsewhere..

Create the file system and restricted access.

dd of=/path/to/secretfs bs=1G count=0 seek=8
chmod 600 /path/to/secretfs

Create a loopback device for the file
losetup /dev/loop0 /path/to/secretfs
Creates the volume, asks you for the key, and then creatss the DM mapping
cryptsetup -y luksFormat /dev/loop0
cryptsetup luksOpen /dev/loop0 secretfs

Pad the file so it generates its random data
dd if=/dev/zero of=/dev/mapper/secretfs

Create the filesystem
mkfs.ext3 dev/mapper/secretfs

# Mount the new filesystem in a convenient location
mkdir /mnt/cryptofs/secretfs
mount /dev/mapper/secretfs /mnt/cryptofs/secretfs

And an encrypted volume is setup.

Now how to unmount:

umount /mnt/cryptofs/secretfs
cryptsetup luksClose secretfs
losetup -d /dev/loop0

Remount:

losetup /dev/loop0 /path/to/secretfs
cryptsetup luksOpen /dev/loop0 secretfs
(enter password when prompted)
mount /dev/mapper/secretfs /mnt/cryptofs/secretfs

Now stuff that i can never remember how to do is grow the volume.

Make sure the volume is closed, unmounted etc.
umount /mnt/cryptofs/secretfs
cryptsetup luksClose secretfs
losetup -d /dev/loop0

Add 20gig to the file
dd if=/dev/zero bs=1G count=20 >> /dev/mapper/secretfs

Remount and reopen part of it

losetup /dev/loop0 /path/to/secretfs
cryptsetup lucksOpen /dev/loop5 EncryptedFS

check the FS first
e2fsck -f /dev/mapper/EncryptedFS

once thats done grow it
resize2fs /dev/mapper/EncryptedFS
and your done. You can now open the volume for use

Trying to use sed in some stuff and i never want to have to write this kinda crap again so..

grep '^D' Filename  | sed -e 's/\(^.\{43\}\)0\(.*\)/\1M\2/g'

That looks for any line in “Filename” that starts with D… then if the 44th character is 0, replace it with an M.
kgo.

grep "^D" Filename | sed -e 's/.*/&XX/' 

That looks for anything in Filename and adds “XX” to the end of the line..
Another Useful link

So I’ve been with Linode.com now for coming up on either 3 or 4 years, I’ve lost track now. I was looking for a remote server that i could play with and run a few services such as mail and maybe a web page or two. So a friend and I bought a linode. Back then the lowest linode you could get was a 32mb linode (32mb of ram), and i think it hat 4gig of HDD space. Not a lot but it was reasonable at 19.95$ a month.

Low and behold 3-4 years later, for the same price linode now offers the Linode 360, with 10Gig of storage off the bat. Every 6 months or so, so far, the team at Linode have managed to excite us all, by giving us free upgrades, the last one 20% ram increase for xmas. Ive been through countless free RAM upgrades, as well as storage, and transfer, and not once has there been a cost associated with it.

The only reason i can say that my uptime is not massively large, is that they keep giving out these upgrades that require us to reboot to take advantage of them. Otherwise i havnt had a much of a problem or downtime, that i havn’t caused myself, or was outside of Linode’s reachs (Datacentre issues).

Thats not to mention the friendliness and helpfulness of the 3 admins, Caker, Tasaro, and Mikegrb who are always available via the ticketing system, and atleast one is generally lurking in the irc channel (#linode on irc.oftc.net). On top of the help from the staff, there are always people arround in the chat channel that are more then willing to help out on just about any linux topic, linode related, or not. That and the forums are always bristling with activity, be it howto guides, customer feedback, announcements or community help requests with specific platform/software packages.

These days i have 3 linodes, 2x 360mb, and a 540mb one, ive never looked back…

With servers in 3 different data centres, full root access, friendly staff and a great community behind them, if your looking for a VPS, you can stop looking. Linode is the place to be.

www.linode.com Got root?

Well i got a few more drives today for my file server, and one of the things i couldnt remember was how to add to the LVM device that exists that stores all my loot.

First off there is
pvdisplay which shows you which physical drives are alocated to what volume group
lvdisplay which gives you details about the actually logical volume, such as its status etc.

Now the part that initially tripped me up was forgetting to create the partition to LVM type on the raid
pvcreate /path/to/device
It’ll show up something like this

pvcreate /dev/md3
Physical volume “/dev/md3” successfully created

Next step is to add it to the existing group
vgextend /path/to/device

channel:/etc/lvm# vgextend vg00 /dev/md3
Volume group “vg00” successfully extended

Now that its in the volume group and its extended that group to incorporate all the devices, you need to extend the logical volume.
lvextend -l +536551 /dev/vg00/lv_space
Where 536551 is the extents listed as free as shwon by pvdisplay

Now that that is done, its just a matter of extending the file system..

So after gearing up to do some changing to my security stuff, i was looking in on suexec. Suexec is great, it runs all processes launched from apache as the person that actually owns them and does a lot of checking to make sure things are what they are meant to be.

The only down side is, its default options are compiled in and cannot be changed. So on Debian this obviously presents a challenge. Never fear! We can rebuild it!!.
Starting off,

cd /usr/src
apt-get build-dep apache2.2-common
apt-get source apache2.2-common

Wait patiently for stuff to download and unzip and then head into the directory it just created. Should be something like this

cd apache2-2.2.3/

Now you want to edit the debian build rules so that it changes the suexec stuff

vi debian/rules

Just near the top there is
AP2_COMMON_CONFARGS

And you will notice that it has a lot of references to suexec.
Just change the options to what you want. In my case i want to change the docroot to be /home/hosting
so mine looks like
--with-suexec-docroot=/home/hosting \

Now to stop apache getting overwritten with the same version next time you update, we need to increment the build number.
dch -i
(Note: if you don’t have this command you need to install devscripts )
edit the build notes at the top to say what you changed, and then exit.

then simply build the package
dpkg-buildpackage
if you now goto the /usr/src directory you will have all the .deb files that you need to install
dpkg -i the ones you need such as apache2.2-common etc, and then run apt-get -f install to install any extra dependencies and your done.

So for the last week or so ive been working solidly on setting up my file server, to the point where it will be a beast of all beasts, and will have multiple different ways of accessing the public and protected data on it, from a variety of systems.

Since I have a mac as my main laptop and so does my other half and some friends, i thought id put some attention into Apple File sharing. The netatalk package provides this under debian, however after some reading and research it seems that the default package does NOT contain the ssl libraries needed to use encrypted passwords for authentication. So after spending some time reading.. I worked out how to do it, the DEBIAN way to get the packages.

First off you want to get the source packages and required dependencies, and the ssl libraries. You will want to be in a directory where you can store the source, as it just by default downloads/extracts to your current working directory.

cd /usr/src
apt-get install openssl cracklib2 libpam-cracklib cracklib2-dev
apt-get source netatalk
apt-get build-dep netatalk

I did get some errors downloading the netatalk source relating to it being unable to verify but im ignoring them due to other issues that I have at the moment.

Next step is to edit the debian build rules file.

cd netatalk-2.0.3
vi debian/rules

it will look something simular to this:

DEB_UPDATE_RCD_PARAMS := defaults 50
DEB_CONFIGURE_EXTRA_FLAGS := \
--with-shadow --enable-fhs \
--with-tcp-wrappers --with-mangling \
--enable-timelord --enable-overwrite \
--with-pkgconfdir=/etc/netatalk \
--with-nls-dir=/usr/share/netatalk/nls \
--disable-logger --enable-srvloc \
--enable-pgp-uam --enable-krb4-uam --enable-krbV-uam
DEB_BUILD_OPTIONS=ssl debuild
##FIXME: Other changes are needed, like enabling DHX plugin
..rest of file..

You need to add the line that says “DEB_BUILD_OPTIONS=ssl debuild”. This will make the system build and compile the package to include encrypted passwords.
Save that file and then exit. Then build the package.

dpkg-buildpackage

A lot of info will fly up the screen, but make sure when it finishes you go back and check over it, and look for a section simular to this


Configure summary:
Install style:
none
AFP:
AFP 3.x calls activated: yes
Large file support (>2GB) for AFP3: yes
DDP enabled: yes
CNID:
backends: cdb dbd last
UAMS:
DHX (PAM SHADOW)
RANDNUM (PAM SHADOW)
Kerberos V
Kerberos IV
PGP
passwd (PAM SHADOW)
guest
Options:
CUPS support: yes
SLP support: yes
tcp wrapper support: yes
quota support: yes
admin group support: yes
valid shell check: yes
cracklib support: yes
dropbox kludge: no
force volume uid/gid: no
Apple 2 boot support: no

You will notice that it says DHX is compiled in. This is the module required for using encrypted passwords.
After that its a simple matter of installing the package.


cd ..
dpkg -i netatalk_2.0.3-4_i386.deb

And off it will go and install it.

You will need to modify the afpd.conf file located in /etc/netatalk to use the new module compiled in.

Simply add this line
-uamlist uams_dhx.so,uams_guest.so,uams_clrtxt.so,uams_passwd.so,uams_gss.so

to the file, arround this area


# Authentication Methods:
# -uampath Use this path to look for User Authentication Modules.
# (default: /usr/lib/netatalk)
# -uamlist Comma-separated list of UAMs. (default:
# uams_guest.so,uams_clrtxt.so,uams_dhx.so)
#
# some commonly available UAMs:
# uams_guest.so: Allow guest logins
#
# uams_clrtxt.so: (uams_pam.so or uams_passwd.so)
# Allow logins with passwords
# transmitted in the clear.
#
# uams_randnum.so: Allow Random Number and Two-Way
# Random Number exchange for
# authentication.
#
# uams_dhx.so: (uams_dhx_pam.so or uams_dhx_passwd.so)
# Allow Diffie-Hellman eXchange
# (DHX) for authentication.
-uamlist uams_dhx.so,uams_guest.so,uams_clrtxt.so,uams_passwd.so,uams_gss.so

You may want to disable a few things that it starts by default though. I didnt want printing support, or the appletalk protocol running so i turned them off by simply editing /etc/defaults/netatalk and changing

ATALKD_RUN=yes
PAPD_RUN=yes
CNID_METAD_RUN=yes
AFPD_RUN=yes
TIMELORD_RUN=no
A2BOOT_RUN=no

to

ATALKD_RUN=no
PAPD_RUN=no
CNID_METAD_RUN=yes
AFPD_RUN=yes
TIMELORD_RUN=no
A2BOOT_RUN=no

That there is your standard setup.
Other files to pay attention to:
/etc/netatalk/AppleVolumes.default : Contains the volumes to share and permissions etc.

Any questions let me know,
Thans

Ok so im playing with raids and what not.. and i thought id quickly document the commands so that when i forget them i can find then easily.

You need the package mdadm

Obviously substituing mdX for the raid number ur setting up.. ie md0 and /dev/hdx? with /dev/hdd1 /dev/hdc2 etc

Basic Commands
Checking status of drive/arrays, displaying all raids and basic information
cat /proc/mdstat

Detailed status of array
mdadm --detail /dev/mdX

Creating raid device
mdadm --create --verbose /dev/mdX --level=5 --raid-devices=2 /dev/hdx? /dev/hdx?

Stoping raid device
mdadm --stop /dev/mdX

Adding a drive:
mdadm /dev/mdX -a /dev/hdx?

Marking a drive as failed (This will break things if your not careful)
mdadm --manage --set-faulty /dev/mdX /dev/hdx?

Removing a drive:
mdadm /dev/mdX -r /dev/hdx?

Forcing a full check of the raid:
echo check >> /sys/block/mdX/md/sync_action

Growing array to include new drive (Y is new number of devices)
mdadm --grow --raid-devices=Y /dev/mdX

Creating Array and File system
Create the initall array with the number of devices you want
mdadm --create --verbose /dev/mdX --level=RAID5--raid-devices=3 /dev/hdx? /dev/hdx? /dev/hdx?

Then format the device to include the file system you want (EXT3 in this case:
mkfs.ext3 /dev/mdX

Then mount your /dev/mdX device and your done.

Expanding your array
To expand your array, to have another drive in it.. you first should force a complete check of the drive so that it doesnt fail while rebuilding.
echo check >> /sys/block/mdX/md/sync_action
That will take a while. Then you add your new device.
mdadm /dev/mdX -a /dev/hdx?
If you do a detailed listing of your raid, you will see it list that you now have a spare. To make the raid grow to include this spare.(Y is the number of devices you will have in the array now)
mdadm --grow --raid-devices=Y /dev/mdX
This will take a while again as it rebuilds the array to include the new drive.

Once this has finished you will want to resize the filesystem to have the new space. Make sure you unmount the drive first.
resize2fs /dev/mdX

Chances are resize2fs will ask you to run a fsck on the drive first. It wont work until its satisifed the drive is stable.

Also after having issues where i accidently forgot to plug a drive back in when i rebuilt the case, i went hunting for a way to not have the raid start and then go into degraded mode if a hdd is not present.

modify the file /etc/init.d/mdadm-raid and change the line that looks like
for line in $($MDADM --assemble --scan --auto=yes --symlink=no 2>&1); do
to
for line in $($MDADM --assemble --scan --no-degraded --auto=yes --symlink=no 2>&1); do
This makes the loading system not assemble any raid that is degraded that isnt started via initrd.

You also have to run dpkg-reconfigure mdadm and make sure that its not loading all devices when it starts, otherwise it will start and load your array which will then be bad.

Update:

Some additional things that i forogt to add.
Sometimes if its really fubar, and it doesnt want to bring it up, or if its in a new host and really being special. You have to create the array again. This sounds counter intuative, but mdadm is smart and as long as its in the same order it can figure it out

mdadm --create --verbose /dev/mdX --level=5 --raid-devices=4 /dev/hda /dev/hdb /dev/hdc /dev/hdd

If a drive is missing, you can force it to create it without it, but make sure you put the word missing in place of the dead disk so it knows the physical order of the array to know where it is.

mdadm --create --verbose /dev/mdX --level=5 --raid-devices=4 /dev/hda missing /dev/hdc /dev/hdd

Also you can examine the contents of a disk itself to know what it thinks the raid table should look like (including the order if you dont remember it)

This will show you the array and what order the disks should be in, and some other useful stuff like what that disk’s blocks think the status of the array is. This information is great if you suffer a major failure on potentially two drives. You can use this information to try and force one of the dead drives back into service, once you know the order…
mdadm --examine /dev/sdb1

Also, to just get some primiative information such as the MD number, and the UUID this will help
mdadm --examine --scan /dev/sdb1