Saturday, 19 November 2016

Raspberry Pi Downloader

So I now have a Pi powered newsgroup indexer and a Pi powered newsgroup server, time for a Pi powered downloader, I'm going to set this up for Torrents and Usenet use, I've added a USB drive to it and mounted it at:


By editing the fstab file with:

sudo nano /etc/fstab

And adding this line to the bottom:

UUID=01669871-a2a3-406e-84a9-f63a9c5fffd0       /home/usernameDownloads        ext4    errors=remount-ro       0       1

You'll have to change the UUID to your drives and username to the user you have on the Pi.

I'm also going to make the downloads folder visible on the network by installing a Samba server.

sudo apt-get install samba samba-common-bin

Make some changes with 

sudo nano /etc/samba/smb.conf

Change the network name and workgroup settings at the top to suit your own use.

And add this to the bottom:
        path = /home/username/Downloads
        browseable = yes
        public = yes
        writeable = yes
        create mask = 0777
        directory mask = 0777

Let's install SabNZB now and get it working:

sudo apt-get install sabnzbdplus

Edit a config file with:

sudo nano /etc/default/sabnzbdplus

It should look something like this when it's finished.

# This file is sourced by /etc/init.d/sabnzbdplus
# When SABnzbd+ is started using the init script, the
# --daemon option is always used, and the program is
# started under the account of $USER, as set below.
# Each setting is marked either "required" or "optional";
# leaving any required setting unconfigured will cause
# the service to not start.

# [required] user or uid of account to run the program as:

# [optional] full path to the configuration file of your choice;
#            otherwise, the default location (in $USER's home
#            directory) is used:

# [optional] hostname/ip and port number to listen on:

# [optional] extra command line options, if any:

Now we make it executable with:

sudo service smbd restart

And then it's time to finish off the configuration by pointing a web browser at:


That's newsgroups sorted, let's move onto torrents:

sudo apt-get install deluged deluge-console deluge-web

Now we're going make deluge create a configuration file and then shut it down afterwards.


sudo pkill deluged

We need to change a files but let's back then up first:

cp ~/.config/deluge/auth ~/.config/deluge/auth.old

At the bottom of the file add the following:


User is the username you're running deluge under, password is the password for accessing the interface and level is 10, so something like:


Now we can run deluge by typing:


And connect to it with the console by typing:


We need to enter the following into the console to allow remote connections:

config -s allow_remote True

config allow_remote


Time now to install the Deluge client on your desktop, you can get it from here. Once it's installed run it and you'll see something like this, depending on your operating system.

Go to preferences, interface and remove the tick from classic mode

Click Ok and then close down the client and reopen it. You should now get the connection manager popping up, click on Add.

And enter the network name of the Pi or it's IP address along with the username and password you added earlier.

You should now see the console where you can control your torrent client

There you go, you now have a nice low powered download computer.

Leafnode News Server

Following on with my setting up of a Raspberry Pi as a Newsgroup Indexer, I've realised that the next big issue is with news servers themselves. I don't really want a load of newsgroup just a few with a reasonable retention. Some of the paid for servers are suffering from missing posts so I've decided to use a Raspberry Pi with a 500GB drive in it and set up my own. If it works well, I'll eventually add on a bigger drive.

I've installed Raspbian Lite and mounted the external drive at /var/news, one problem is that the drive doesn't seem to mount at boot, this has been fixed by modifying the cmdline.txt file, run:

sudo nano /boot/cmdline.txt

And add rootdelay=5 to the end.

Once you've done all that let's bring the machine up to date with:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get autoremove

and then restart it with:

sudo shutdown -r now

Once it's rebooted, installation is fairly straightforward with:

sudo apt-get install leafnode

You get asked to enter the details of a news server, I've got Virgin Media here so it's and a permanent connection.

Then the fun begins, you're going to need to edit your hosts file with:

sudo nano /etc/hosts

Edit the line that says machinename

so that it shows nachinename.domain machinename

You can't use localdomain here, so just put your Samba workgroup domain in, then we need to edit another two files so we can connect on the local network:

sudo nano /etc/hosts.allow

#-- leafnode begin
#-- leafnode end

Edit this so it looks like

#-- leafnode begin
#-- leafnode end

Changing the to whatever your IP address range is and then:

sudo nano /etc/hosts.deny

#-- leafnode begin
leafnode: ALL
#-- leafnode end

Comment out the middle line so it looks like this:

#-- leafnode begin
# leafnode: ALL
#-- leafnode end

Now it's time to fetch a list of newsgroups:

sudo fetchnews -vvv

You can now connect to the server with news reader, I'm going to use Pan but there are loads out there, this is how you do it with Pan

Subscribe to a newsgroup and then go and request the latest headers, this will only give one leafnode entry at the moment, now we want to change some settings before we go any further, just enter:

sudo nano /etc/news/leafnode/config

I'm going to change the expire entry at the top to 400, that's just over a year

Going down the screen I'm changing the initial fetch to 1000, keep going down the screen and set maxfetch to 2000, maxcrosspost to 10 and maxage to 1000. If you're on a slow connection it may be worth you dropping some of these settings

We then run:

sudo fetchnews -vvv

This time it will populate the group(s)

You may find that your current news server doesn't hold all the groups you want so you can edit the leafnode config file again and add some other servers in, here's some you can add in;;

server =
username = username
password = password

You can sign up for a free account with XSUsenet by following this link.

# Free Servers

server =

server =

server =

server =

Now run sudo fetchnews -vvv again and it will pull down a list of groups from these servers, make sure that you ask your usenet client to refresh the groups list.

Last thing we have to do is make it update the groups by itself, there is a file in the leafnode folder that you can use but I've done it this way and it works for me:

sudo nano /bin/getnews

Paste the following into it:

texpire -v
fetchnews -vvv

Let#s make it executable with:

sudo chmod +x /bin/getnews

And we'll get cron to run this every 15 minutes with:

sudo crontab -e

Paste this into it:

*/15 * * * * /bin/getnews

Then save it, give it a final reboot and you're all sorted, you're very own news server. You can't add a username and password for access as far as I'm aware so it's not a good idea to make it accessible from the Interwebs.

If you reboot at some point the lockfile may stop leafnode updating, I've created another file called clearlock with:

sudo /bin/clearlock

This goes into it:

rm /var/spool/news/leaf.node/lock.file 

Again, we make it executable with:

sudo chmod +x /bin/clearlock

And we make it run on boot with:

sudo crontab -e

Paste this in.

@reboot /bin/mtclearlock

And we're all done

Sunday, 6 November 2016

Newznab Pi

A lot of the newsgroup indexing sites are either being shut down or are becoming invite only which is a shame as there's a lot of useful stuff in the groups that isn't only illegal downloads. There's quite a few Linux, freeware and shareware groups out there. 

It's time to convert a Raspberry Pi into a usenet indexer, I tried this with an original Pi and it was far too slow but hopefully the model 2 will do the job. All configuration is done from a Linux desktop, if you haven't got an ssh command then you'll need to get something like Putty.

So what do we need, obviously a Raspberry Pi model 2 in a box, something like this will do.

An external hard drive of some sort, I've used an old 320Gb and got hold of a case from Amazon, it's
nothing exciting but it works, I ordered the case on Saturday evening and it arrived on Sunday lunchtime, good going there Amazon.

A USB hub also makes life easier, nice cheap one from Maplin. You're also going to need an ethernet lead to connect it to your router, £1 in Poundworld.

Some coffee and a sausage inna bun will help.

It's time to make a start, put it all together but don't worry about connecting the external drive yet, we need to get hold of Raspbian Jessie Lite from here. Extract it and then copy it to a micro SD card, easiest way is to use a cross platform app called Etcher

Once it's finished, put the card into the Pi and give it some power. Have a look at the connected devices in the router config and ssh into the pi with ssh pi@ or whatever yours says, password to connect is raspberry. We now run:

sudo raspi-config

You should see this:

Scroll down to advanced, then down to update, once that's completed go back down to advanced options and put in your own hostname, mine is called Orac as the Pi is in a clear case just like Orac in Blakes 7, under advanced we want to select boot to console requiring user to login and enable ssh. Once last thing under advanced is memory split, set this to 16 to give as much RAM to the running of the Pi as possible. Last thing we do is select Expand Filesystem at the top.

Select finish and wait for a reboot, you should now be able to connect with 

ssh pi@orac

 or whatever you've called your pi.

Now we create a normal user with

sudo adduser username

then add them to the superuser group with

sudo adduser username sudo

Reboot with 

sudo shutdown -r now 

Now you should be able to login with ssh orac or ssh username@orac.

Let's remove the default Pi login with

sudo deluser pi

We need some helpful utilities now, so let's enter:

sudo apt-get install rsync mc

And now it's time to connect up the external drive.

Type dmesg at the command line and it should come up as /dev/sda or something similar, we now need to partition it so we enter:

sudo fdisk /dev/sda or whatever yours came up as.

Then D to delete any existing partitions, then N to create a new one, followed by P for primary partition, then enter 3 times followed by w to write the changes. There may be other options here depending on how many partitions there were on the drive when you started.

Next we have to format the drive with:

sudo mkfs.ext4 /dev/sda1

Now create a temporary mount point with 

sudo mkdir /mnt/temp

Then copy the filesystem to the external drive with:

sudo rsync -axv / /mnt/temp

Let's back some config files with:

sudo cp -R /boot /boot.orig

Time to edit some  stuff to allow us to boot from the external drive:

sudo nano /boot/cmdline.txt

The original looks like this:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

It's all on one line, we need to change the root=/dev/... bit to read root=/dev/sda1 or whatever your USB stick shows, we also need to add a delay to the end with rootdelay=5, it should look like the one below in the end.

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline rootwait rootdelay=5

Save it and then we need to edit fstab with:

sudo nano /etc/fstab

The following entry needs to be added:

/dev/sda1    /   ext4    defaults,noatime  0       1

And the line referring to the sd card needs to be commented out, mine looks like this now

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
# /dev/mmcblk0p2  /               ext4    defaults,noatime  0       1

/dev/sda1    /   ext4    defaults,noatime  0       1

We now should be running from the external drive, type df -h to check the space, so now we move onto getting Newznab installed, there's a few utilities we need that don't seem to be in the raspbian repository, I've precompiled these so you can get them with these commands:




Let's get them installed and the system updated now

sudo apt-get update

sudo apt-get upgrade -y

sudo dpkg -i *.deb

Let's make some folders and set permissions:

sudo mkdir /var/www
sudo mkdir /var/www/newznab
sudo chmod 0777 /var/www/newznab

And now install some stuff

sudo apt-get install php5 php5-dev php-pear php5-gd php5-mysql php5-curl mysql-client-5.5 libmysqlclient-dev apache2 mysql-server-5.5 lame mediainfo

Make sure you put in a good strong mysql passqword if you're going to get access to this from the Interwebs.

Now we make some changes to the php configuration file:

sudo nano /etc/php5/cli/php.ini

Under the resource limits section we need to change the maximum execution time to 120:

max_execution_time = 120

Then under Module Settings we need to change the time zone settings, in my case I'm the UK so I'm going to change it to:

date,timezone = Europe/London

Save that and then time to edit another one:

sudo nano /etc/php5/apache2/php.ini

Under the resource limits section we need to change the maximum execution time to 120:

max_execution_time = 120


memory_limit = -1  

Then under Module Settings we need to change the time zone settings, in my case I'm the UK so I'm going to change it to:

date,timezone = Europe/London

Now we create the Apache config file for Newznab

sudo nano /etc/apache2/sites-available/newznab.conf

And paste the following into it:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName localhost

DocumentRoot /var/www/newznab/www
ErrorLog /var/log/apache2/error.log
LogLevel warn
Change the port if you want it to run on a different port.

One more change to the Apache configuration file before we go on:

sudo nano /etc/apache2/apache2.conf

Find this bit

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted

And change it to this:

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted

We now need to get hold of Newznab itself, you can get it from here. Newznab classic works but there are some limitations, if you are serious just pay the money for a better product and you get free upgrades too.

Get the file downloaded and then we need to extract it and copy it to the right place, 

As I have a paid for version I'm going to use a slightly different approach, for this you need to install subversion with the following command:

sudo apt-get install subversion

Then to get Newznab you enter the following command:

sudo svn co svn:// /var/www/newznab

At this point you will be asked for the root password for your system and the SVN login details that have been emailed to you by the Newznab team.

Now it's time to activate the changes:

sudo a2ensite newznab
sudo a2dissite 000-default
sudo a2enmod rewrite
sudo service apache2 restart

Change some folder permissions with:

sudo chmod 777 /var/www/newznab/www/lib/smarty/templates_c
sudo chmod 777 /var/www/newznab/www/covers/movies
sudo chmod 777 /var/www/newznab/www/covers/music
sudo chmod 777 /var/www/newznab/www/covers/tv
sudo chmod 777 /var/www/newznab/www
sudo chmod 777 /var/www/newznab/www/install
sudo chmod 777 /var/www/newznab/nzbfiles/
sudo chmod 777 /var/www/newznab/www/covers/anime
sudo chmod 777 /var/www/newznab/www/covers/tv

And that's it for the moment, you now need to finish setting up Newznab from a browser with:


Once you've done all that, run

sudo chmod 0777 /var/www/newznab/nzbfiles/tmpunrar

And install phpmyadmin with:

sudo apt-get install phpmyadmin

Go to the site admin options and edit site, it works much better if you put in your own Rotten Tomatoes api, your own Amazon settings and your own Newznab ID, the section that asks for the path to some files should show this:


Once that's all completed, go to the View Groups option and make some groups active.
 Then it's back to the command line with

sudo -i

cd  /var/www/newznab/misc/update_scripts

Then run 

php update_binaries_threaded.php

after that it's

php update_releases.php 

That should start populating the indexer, it's up to you how you want to run this, you can it manually but I run a loop with a 10 minute pause each time it restarts.

I've made a file with 

sudo nano /bin/newznabrun

Put this into it

while :

# Beginning Update
cd /var/www/newznab/misc/update_scripts/
php update_binaries_threaded.php
php update_releases.php

        echo "Press [CTRL+C] to stop.."
        sleep 360


Save the file and then run

sudo chmod +x /bin/newznabrun 

You can then type sudo newznabrun to get it going.

That's it all done, hope it all works for you.

Big thanks to the howtogeek website for help with these instructions, I've added a few bits and pieces to get to this stage.

Sunday, 10 July 2016

More Ubuntu Woes, back to the Future with PCLinuxOS.

I'm going to admit that I actually like the Unity desktop, there, I feel better now. I've used Ubuntu 14.04 for a while but as it gets older and you add more and more ppas to keep the apps up to date it's become slower and less stable, the latest issue has been after a load of updates I rebooted into a 1024x768 screen, easily sorted by installing the AMD drivers.

I decided once more to try and install 16.04 or one of the Mint derivatives, each one gave me the same issue, just a blank screen when trying to install. I wasn't going to give up, during install I pressed the down arrow key when the initial screen appeared, select UK then F6 and selected the nomodeset option. I could now install Ubuntu 16.04. I rebooted and got my lovely 1024x768 resolution, the fix to get this sorted was to modify the grub file to give me a 1440x900 screen with slow software video drivers. I looked up the installation of the AMD drivers, followed the instrctions, rebooted and it died in the most spectacular fashion with lots of lovely dots and so on.

I've given up, this was what Windows 3.1 was like trying to get the right resolution, it shouldn't need this much messing around. I decided to abandon Ubuntu and booted up Fedora Mate, no bottom or top bar visible on Fedora 24 although it worked on 23, the only other distro I had on my pendrive was PCLinuxOS, the latest preview version, it's pretty much a beta. I booted this, everything was there, I installed it, everything worked, it had automatically installed the right drivers for me. Even the bluetooth settings worked, on Ubuntu 14.04 you had to manually change the device name by editing a file. The most impressive thing is that the num lock worked and was turned on when I powered up, I didn't have to do this manually every time I started up the machine.

There's a few little quirks, you need to go into the Configure Your Computer option and select the Setup a network interface option to set the hostname, this requires a reboot.

You also need to go into the software centre and run the localisation utility to change your language, by default it's US, another reboot is needed, there's no sudo but this makes the machine more secure.

Installing Virtual Box is a breeze, you click on the Virtual Box Manager link, it installs the non free version so USB works, no messing round, it just works. 

The other good thing is that it's a rolling release, no need to re-install, you download the updates and you have the latest version. 

I used to use PCLinuxOS when Mandriva became to buggy, it never really let me down and it hasn't again, it uses rpm files but synaptic to manage them, if you want to get into Linux, try it.

Tuesday, 21 June 2016

OpenVPN Server on Debian Jessie

It was time to setup a way of getting secure access to my network while away and also to get a secure connection to the Interwebs while away from home, it also means that I can watch BBC Iplayer as if I'm at home.

I'm installing this on an old machine running Debian Jessie, it should work on a Raspberry Pi too, the only things I have installed during the initial setup process as the standard Debian utilities and ssh server so I can do everything remotely.

I've found lots on instructions out there but the one from this website was the easiest to follow, I've modified it slightly to make it easier to get at the keys.

I've modified a few things myself

First thing is to ensure we are up to date, lets switch to root for the install



apt-get update
apt-get upgrade

Time to start installing stuff

apt-get install openvpn easy-rsa

Then copy some example files over to make the job easier

cp -r /usr/share/easy-rsa/ /etc/openvpn
mkdir /etc/openvpn/easy-rsa/keys

Now we edit the certificate variables

nano /etc/openvpn/easy-rsa/vars

# These are the default values for fields
# which will be placed in the certificate.
# Don't leave any of these fields blank.
export KEY_COUNTRY="changeme"
export KEY_PROVINCE="changeme"
export KEY_CITY="changeme"
export KEY_ORG="example"
export KEY_EMAIL=""
export KEY_OU="changeme"

# X509 Subject Field
export KEY_NAME="server"

Time to generate some stuff and go and have a coffee, on a Pi, this may take some time

openssl dhparam -out /etc/openvpn/dh2048.pem 2048

Now we make the server certificate keys:

cd /etc/openvpn/easy-rsa
. ./vars
./build-key-server server

Let's copy them to where they belong

cp /etc/openvpn/easy-rsa/keys/{server.crt,server.key,ca.crt} /etc/openvpn

Now time to make some changes to the network settings:

echo 1 > /proc/sys/net/ipv4/ip_forward

And let's make the changes permanent with:

nano /etc/sysctl.conf

Look for the following bit:

# Uncomment the next line to enable packet forwarding for IPv4
# net.ipv4.ip_forward=1

Then remove the # from the second line so it looks like this:

# Uncomment the next line to enable packet forwarding for IPv4

Now we make the server config file:

nano /etc/openvpn/server.conf

Paste this lot into the empty file, this will run the VPN server on port 1194.

port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key 
dh dh2048.pem
ifconfig-pool-persist ipp.txt
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS"
keepalive 10 120
cipher AES-128-CBC
user nobody
group nogroup
status logs/status.log
log-append logs/openvpn.log
verb 3

Now we make the log files:

mkdir -p /etc/openvpn/logs
touch /etc/openvpn/logs/{openvpn,status}.log

And let's do some firewall configuration:

iptables -t nat -A POSTROUTING -s -o eth0 -j MASQUERADE

Now let's restart the server to put the changes into place:

systemctl restart openvpn@server.service

Now the original instructions came with a script file to help you create new keys for each user and device, pointless changing it.

nano /etc/openvpn/

Paste this lot in:



# Generating key
echo "Generating key for user ${username}"
cd /etc/openvpn/easy-rsa/
source vars && ./pkitool ${username}
cp /etc/openvpn/clients/.tmp/.tmp.ovpn /etc/openvpn/clients/.tmp/${username}.ovpn
echo "Done"

# Adding ca certificate to ovpn client configuration file
echo "Adding ca certificate to ovpn client configuration file"
echo "<ca>" >> /etc/openvpn/clients/.tmp/${username}.ovpn
cat /etc/openvpn/easy-rsa/keys/ca.crt | grep -A 100 "BEGIN CERTIFICATE" | grep -B 100 "END CERTIFICATE" >> /etc/openvpn/clients/.tmp/${username}.ovpn
echo "</ca>" >> /etc/openvpn/clients/.tmp/${username}.ovpn
echo "Done"

# Adding user certificate to ovpn client configuration file
echo "Adding user certificate to ovpn client configuration file"
echo "<cert>" >> /etc/openvpn/clients/.tmp/${username}.ovpn
cat /etc/openvpn/easy-rsa/keys/${username}.crt | grep -A 100 "BEGIN CERTIFICATE" | grep -B 100 "END CERTIFICATE" >> /etc/openvpn/clients/.tmp/${username}.ovpn
echo "</cert>" >> /etc/openvpn/clients/.tmp/${username}.ovpn
echo "Done"

# Adding user key to ovpn client configuration file
echo "Adding user key to ovpn client configuration file"
echo "<key>" >> /etc/openvpn/clients/.tmp/${username}.ovpn
cat /etc/openvpn/easy-rsa/keys/${username}.key | grep -A 100 "BEGIN PRIVATE KEY" | grep -B 100 "END PRIVATE KEY" >> /etc/openvpn/clients/.tmp/${username}.ovpn
echo "</key>" >> /etc/openvpn/clients/.tmp/${username}.ovpn

mkdir -p /etc/openvpn/clients/${username}
mv /etc/openvpn/clients/.tmp/${username}.ovpn /etc/openvpn/clients/${username}/${username}.ovpn
cp /etc/openvpn/easy-rsa/keys/${username}.{crt,key} /etc/openvpn/clients/${username}
cp /etc/openvpn/easy-rsa/keys/ca.crt /etc/openvpn/clients/${username}

cd /etc/openvpn/clients; tar -jcf ${username}.tar.gz ${username}/

chmod 0777 -R /etc/openvpn/clients

echo "Done"

echo "

            Configurations are located in /etc/openvpn/clients/${username}


                        Download friendly version with:

         'scp root@`hostname -f`:/etc/openvpn/clients/${username}.tar.gz .'


exit 0

Save it and then make it executable with:

chmod +x /etc/openvpn/

Next we have to create the template file for this to use:

mkdir -p /etc/openvpn/clients/.tmp/

nano /etc/openvpn/clients/.tmp/.tmp.ovpn

Paste this in, change for your external IP or server address

verb 1
dev tun
proto udp
port 1194
remote 1194 udp
remote-cert-tls server
resolv-retry infinite
cipher AES-128-CBC

Now, let's make some keys:

cd /etc/openvpn/

replace username with your username, I'm going to install this onto an S5 so it will be freds5 or something.

./ username

To make it easy to get the files off the server and onto my device, I've decided to install Samba and setup the client keys folder as a Windows share, this is how this is done.

apt-get install samba samba-common

Once it's finished we edit the Samba config file:

nano /etc/samba/smb.conf

Change the workgroup name at the top of the file and you can also add:

netbios name = servername under it if you want.

Add the bottom add the following

        path = /etc/openvpn/clients/
        browseable = yes
        public = yes
        writeable = no

Restart the server with:

service smbd restart

Just got to change the folder permissions to make sure we can get the files off:

chmod 0777 -R /etc/openvpn/clients

Onto my phone now, I've installed OpenVPN Connect from the play store, then I've copied the files from the Windows share into dropbox, then saved them into a folder on the phone called VPN, you could just install a file browser and do the same.

Then import the .ovpn file into OpenVPN connect and click on connect.

One last thing, make sure you give the server a static IP and forward port 1194 on the router.