Showing posts with label System / Network Administration. Show all posts
Showing posts with label System / Network Administration. Show all posts

Saturday, September 11, 2010

Mythbuntu and Tivo Premier

I was using Windows 7 for my secondary media server and was very frustrated with its performance when it came to Tivo desktop,  which is the WORST part of Tivo.  Tivo should be able to stream video to another server,  and there are many opensource applications to do this but of course Tivo has other things to focus on like a terrible new interface and a buggy new OS, but I digress.   I love my Tivo but I don't want to have to buy two of them to watch shows in a different room.  So I left Windows 7 and its lackluster performance for Mythbuntu,  now I just needed to get my Tivo programs to Mythbuntu.  Thanks to the work from the Tivodecode Project and Ed Salisbury's Perl Script I was able to replicate the features of TivoToGo in Linux with a much better playback and keep everything in Boxee.  Here is Ed’s Script:


#!/usr/bin/perl
# TiVo Dump
# Copy TiVo programs from a Series3 TiVo
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * curl
# * tivodecode
#
# External Perl Modules Required:
# * Net::TiVo
# * XML::Simple
# * Text::Unidecode;
#
# Usage:
# tivo_dump
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share - to copy, distribute and transmit the work
#   * to Remix - to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;
use Net::TiVo;
use XML::Simple;
use utf8;
use Text::Unidecode;

sub fix_chars($);

# User-Configurable Variables
my $HOST = "";      # Hostname/IP of the TiVo
my $MAK = "";       # The Media Access Key of the TiVo
my $VIDEO_DIR = ""; # Where the programs get saved
my $COPY_SUGGESTIONS = 0; # If you want to copy "Suggested" programs, set this to 1

my $USER = "tivo";
my $TMPFILE = "/tmp/$$.xml";
my $PROGRAMS_FILE = "/home/username/.tivo_programs";
$|++;

# External Utilities
my $CURL = "/usr/bin/curl";
my $TIVODECODE = "/usr/local/bin/tivodecode";

my @PREV;

# Connect to the TiVo
print "Connecting to TiVo at $HOST... ";
my $tivo = Net::TiVo->new(host => $HOST, mac => $MAK);
my @folders = $tivo->folders();
if (@folders)
{
    print "OK\n";
}
else
{
    print "FAIL\n";
    exit();
}

# Load the file that has the previously saved programs
open(IN, $PROGRAMS_FILE);
while ()
{
    chop();
    push(@PREV, $_);
}
close (IN);

# Go through each folder on the TiVo
foreach my $folder (@folders)
{
    foreach my $item ($folder->{'xmlref'}{'Item'})
    {
        foreach my $video (@$item)
        {
            # Only process videos, not folders
            if ($video->{'Links'}{'Content'}{'ContentType'} eq "video/x-tivo-raw-tts")
            {
                # Choose video type based on the icon
                if ($video->{'Links'}{'CustomIcon'} && $video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:suggestion-recording" && !$COPY_SUGGESTIONS)
                {
                    next;
                }
                if ($video->{'Links'}{'CustomIcon'} &&
                   ($video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:in-progress-transfer" ||
                    $video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:in-progress-recording"))
                {
                   next;
                }

                # Get program and episode titles
                my $program_title = $video->{'Details'}{'Title'};

                if ($video->{'Details'}{'EpisodeTitle'})
                {
                    $program_title .= " - " . $video->{'Details'}{'EpisodeTitle'};
                }

                # Get Program ID and Video URL
                my $video_url = $video->{'Links'}{'Content'}{'Url'};
                my $program_id = $video->{'Details'}{'ProgramId'};

                if (!$program_id)
                {
                    next;
                }

                # If previously copied, skip
                if (grep /^$program_id$/, @PREV)
                {
                    print "Skipping $program_title.\n";
                    next;
                }

                # get details XML file
                print "Getting details for $program_title... ";
                my $details_xml = $video->{'Links'}{'TiVoVideoDetails'}{'Url'};

                system("$CURL --digest -s -k -u $USER:$MAK -c /tmp/cookies.txt -o $TMPFILE \"$details_xml\"");
                if (-f $TMPFILE)
                {
                    print "OK\nProcessing details file... ";
                    my $xml = XML::Simple->new();
                    my $doc = $xml->XMLin($TMPFILE);
                    my %meta;
                    my $filepath;
                    my $filename;

                    # Get rating and convert to proper form
                    $meta{'tvRating'} = $doc->{'showing'}{'tvRating'}{'content'};
                    if ($meta{'tvRating'})
                    {
                        if ($meta{'tvRating'} eq "Y_7") { $meta{'tvRating'} = 'x1'; }
                        elsif ($meta{'tvRating'} eq "PG") { $meta{'tvRating'} = 'x4'; }
                        elsif ($meta{'tvRating'} eqfilepath = "$VIDEO_DIR/$series_title";
                        $filename = "$filepath/$series_title - $episode_number - $episode_title";
                    }
                    elsif ($series_title && $episode_title)
                    {
                        $filepath = "$VIDEO_DIR/$series_title";
                        $filename = "$filepath/$series_title - $episode_title";
                    }
                    elsif ($title)
                    {
                        $filepath = "$VIDEO_DIR/$title";
                        $filename = "$filepath/$title";
                    }
                    else
                    {
                        $filepath = "$VIDEO_DIR";
                        $filename = "$filepath/Unknown";
                    }
                    print "OK\n";

                    unless (-d $filepath)
                    {
                        print "Path $filepath doesn't exist, creating... ";
                        mkdir($filepath);
                        if (-d $filepath)
                        {
                            print "OK\n";
                        }
                        else
                        {
                            print "FAIL\n";
                            exit;
                        }
                    }

                    # Get the video with curl
                    print "Getting video... ";
                    system("$CURL --digest -s -k -u $USER:$MAK -c /tmp/cookies.txt -o \"$filename.tivo\" \"$video_url\"");
                    if (-f "$filename.tivo")
                    {
                        my $filesize = (stat("$filename.tivo"))[7];

                        if ($filesize > 0)
                        {
                            print "OK\n";

                            # Convert to MPG
                            print "Converting video to MPG format... ";
                            system("$TIVODECODE -m $MAK -o \"$filename.mpg\" \"$filename.tivo\" > /dev/null 2>&1");
                            if (-f "$filename.mpg")
                            {
                                print "OK\n";
                                unlink "$filename.tivo";
                                open(OUT, ">>$PROGRAMS_FILE");
                                print OUT "$program_id\n";
                                close(OUT);
                            }
                            else
                            {
                                print "FAIL\n";
                                exit();
                            }

                            # Output metadata file
                            print "Outputting metadata... ";
                            open (OUT, ">$filename.mpg.txt");

                            foreach my $key (keys %meta)
                            {
                                if ($meta{$key})
                                {
                                    if ($meta{$key} =~ /^ARRAY/)
                                    {
                                        foreach my $item (@{$meta{$key}})
                                        {
                                            unless ($item =~ /^HASH/)
                                            {
                                                print OUT "$key : " . fix_chars($item) . "\n";
                                            }
                                        }
                                    }
                                    else
                                    {
                                        unless ($item =~ /^HASH/)
                                        {
                                            if ($key eq "originalAirDate")
                                            {
                                                print OUT "$key : " . $meta{$key} . "\n";
                                            }
                                            else
                                            {
                                                print OUT "$key : " . fix_chars($meta{$key}) . "\n";
                                            }
                                        }
                                    }
                                }
                            }
                            unlink($TMPFILE);
                            close(OUT);
                            print "OK\n";
                        }
                        else
                        {
                            print "FAIL\n";
                        }
                    }
                    else
                    {
                        print "FAIL\n";
                        exit();
                    }
                }
                else
                {
                    print "FAIL\n";
                    exit();
                }
            }
        }
    }
}

# Convert any offending characters
sub fix_chars($)
{
    my ($data) = @_;

    $data = unidecode($data);

    $data =~ s/\:/ -/g;
    $data =~ s/\//-/g;
    $data =~ s/\\/-/g;
    $data =~ s/\?/-/g;
    $data =~ s/\*/-/g;

    return $data;
}


then just add it to your cron
0 0,6,12,18 * * * /usr/local/bin/tivo_dump >/dev/null 2>&1
An then your now playing or now playing and suggestions will all be copied to your Linux box every x hours.

Friday, September 3, 2010

How to program your MCE remote

To program buttons on your MCE remote control.

  1. Place the MCE remote control head to head (2 to 3 inches apart) with the remote control from which it is learning.
  2. On the MCE remote control press and hold the DVD MENU and OK buttons at the same time until the remote control lights turn off (2 seconds). The remote is now in learning mode.
  3. On the MCE remote control press and release the TV, VOL +, VOL – button, depending on which command you want it to learn. The MCE remote control lights blink once to confirm the selection.
  4. On the remote control that is teaching the command, press and hold the button that you want the corresponding button on the MCE remote control. The MCE remote control lights blink twice to confirm the selection. If the remote does not learn the command, the lights blink quickly four times. To try again repeat steps 1 through 4.
  5. Repeat steps 1 through 4 for the other buttons that can learn commands.

Mythtv revisited with Mythbuntu

image 4 years ago I built a HTPC using Fedora and the experience was miserable,  I had given up on a linux HTPC and moved to Windows and was relatively happy until my S.M.A.R.T. drive reported impending doom.  I had to save my data and rebuilt my server,  which made me want to revisit RAID 1 (mirroring) and Linux.  From my experience, linux was going to be an uphill battle so I picked a path of least resistance with Mythbuntu (http://www.mythbuntu.org/)  and I was very pleasantly surprised.  I have a 3ware raid controller which is very easily supported across all linux platforms (3ware 8006) with a ASUS M2N-E motherboard with a nvidia graphics card (Gigabyte 8400GS 512MB GV) .  I recommend a fanless graphics card. 

 

I installed Mythbuntu and my mce remote and everything worked like a charm out of the box, AMAZING!  I had to turn off the display blanking and uninstall the xscreensaver but that was relatively easy.  Just add the following lines to your xorg.conf

/etc/X11/xorg.conf

Section "ServerFlags"
Option "blank time" "0"
Option "standby time" "0"
Option "suspend time" "0"
Option "off time" "0"
EndSection

I also installed Miro, Subsonic, and qbittorrent.  Miro for mobile video streaming, Subsonic for mobile audio,  and qbittorrent for rss torrent downloads.

I dont use Mythtv instead I use boxee which I launch from mythtv via the following addition to the mythtv menu

/usr/share/mythtv/themes/defaultmenu/mainmenu.xml

After:
<mythmenu name="MAIN">

Put:

<button>
<type>MENU_BOXEE</type>
<text>Boxee</text>
<action>EXEC /opt/boxee/run-boxee-desktop</action>
</button>

On day two I am very pleased with the progress of Mythbuntu as Boxee box.

Saturday, July 10, 2010

Everyone has problems : Twitter and Reddit

For those of you out there that just expect everything to work on the internet,   here is a great internal view to some of the hard challenges that face some of the largest sites on the internet. 

Twitter listed its challenges as of late:
“Twitter has experienced several incidences of poor site performance and a high number of errors due to one of our internal sub-networks being over-capacity.
We're working hard to address the core issues causing these problems --more on that below-- but in the interests of the open exchange of information, wanted to pull back the curtain and give you deeper insight into what happened and how we're working to address this week's poor site performance.
What happened?
In brief, we made three mistakes:
* We put two critical, fast-growing, high-bandwith components on the same segment of our internal network.
* Our internal network wasn't appropriately being monitored.
* Our internal network was temporarily misconfigured.
What we're doing to fix it
* We've doubled the capacity of our internal network.
* We're improving the monitoring of our internal network.
* We're rebalancing the traffic on our internal network to redistribute the load.”
(http://engineering.twitter.com/2010/06/perfect-stormof-whales.html)

Reddit has also had some huge challenges with traffic:
“We've been kinda bummed at reddit these days. It seems like every week something comes up that slows performance to a crawl or even leads to a total site outage. And we almost never get a chance to release new features anymore.
Our four engineers -- KeyserSosa, jedberg, ketralnis, and myself -- are working full time (plus many evenings and weekends and sometimes even the middle of the night) just to keep things going. Perhaps we're doing it wrong: there might be ways to optimize our code, or technologies that could allow us to work more efficiently, but we're too busy to investigate these things, or to migrate to the ones that look promising. It becomes a vicious cycle.
The bottom line is, we need more resources.”
(http://blog.reddit.com/2010/07/reddit-needs-help.html)

Tuesday, April 21, 2009

Simple cron script to watch a firefox kiosk

Put this in your crontab -e:

* * * * * /user/cron.sh

then in /user/cron.sh put

#!/usr/bin/bash
TEST=`ps ax | grep firefox | grep -v grep | grep -v /usr/bin/firefox | wc -l`
if [ $TEST = 0 ] ; then
/usr/bin/firefox &
fi


and make sure it is set to chmod 755 /user/cron.sh

Tuesday, December 2, 2008

How to set an auto update script for your dynamic dns name for asterisk/freepbx

If you have ever setup an asterisk box then you know some changes are in the configuration. One of those challenges is being behind a nat or router. The issue usually arises when you use the ivr and you dont receive DTMF tones to your ivr. I use ViaTalk for service and their support staff clued me into the problem. I had to externhost=my.dyndns.com in your sip_nat.conf does not solve this problem, or at least didnt for me so I had to write my own dynamic dns update script using php only because it was quick but you can do this is bash probably just as easily.





Thursday, June 5, 2008

How to make your Dell m1210 into a Mac (dual boot with Vista)

First off, I take no responsibility for any damage done to your computer, I am just explaining a possible way to get mac osx running on a dell m1210, if you break your computer, dont cry to me. Also buy osx, if you are from apple, make osx for regular pc's so I dont have to do this.


First off you are going to need some things:


-Utorrent here

-IMG burn here

-The OSX86 driver disk for the m1210 (only if you plan to upgrade to 10.5.3) Download file

-Kalyway OSX ISO 10.5.2 Download file

-Kalyway Combo Update 10.5.3 (only if you plan to upgrade to 10.5.3) Download file

-Your Vista install dvd


Things you should know now before starting...

-Scary things will happen (drives will have problems, but hopefully this will walk you through that) but you should be fairly os savvy before using this, I dont go into a mass of detail but if you email me I will add more

-if you have the nvidia card, your computer or monitor cant go into sleep mode, it just wont work

Check this blog for update on Nvidia (Nvinstaller.com)

-if you have the 1394 wifi card you will have to follow the iwidarwin project (iwidarwin) which had stopped as of the time I wrote this , I recommend the Dell 1390, it works out of the box

- this process causes osx to load by default, if you dont want that, dont follow this ( I will work on a fix for this but I dont have one yet)>

- Audio does not work without a different how to




Ok first burn all your ISO's and the pkg updates and osx drivers to a cd/dvd.

With the Gparted ISO:

If you are shrinking your drive, first off find the drive that has your Vista install, this is usually the largest.

#1 If you have a spare 15-30gig partition skip to step #2

-Load Gparted and delete all the other partitions but the vista partition, (NOTE: I dont use the media direct, if you do, dont follow this, because this will delete media direct)

- move (using resize/move) the vista partition to the very front of the drive and allow for 15-30gigs at the end of the drive (for macos) (this takes forever so get dinner and a movie)

#2 make your second partition (the empty one) a NTFS partition with the label "OSX"

#3 set the attributes to the partition to bootable or boot

(now vista wont load but dont worry about that yet)

With the Kalyway 10.5.2 ISO :

Once the DVD boots it will ask you for the language, then tells you to agree to kalyways terms, click agree, then their will be a bar at the top that says Utilities


-click utilities -> disk utility

-pick your "OSX partition"

-click Erase

Choose volume format macos extended journaled

label it OSX

- click Erase

-once completed close the window

-in the main window now it will ask you to choose and install location

choose the osx partition

-next it will ask you to install (DONT)>

-Click CUSTOMIZE

under this use all the default settings

- Graphics

if you want the computer to sleep correctly choose INTEL GMA

otherwise if you have the Nvidia card choose nvida

- Network Drivers->Network Cards

Choose AppleBCM440xEthernet

Under Patches -> leave the SMBIOS all as the default

and choose the rest (boot_with_cpus=1 is the most important)

then Close and install

OSX should boot and work fine

next getting vista working

- get your vista boot disk

- load the vista install

- click repair

- under repair it wont find anything

-click next or whatever and it will show a page one option is cmd prompt

-click command prompt

-type the following

diskpart

select disk 0

list part

(usually osx is part 2)

select part 1

active

go back to the window with the utilities and hit repair

it will say your disk has been repaired

reboot

vista should now come up

after vista boots (it may need to boot twice to repair completely)

then go to command prompt

type the following

diskpart

select disk 0

select part 2

active



Now you are set... when you reboot your system, the darwin boot loader will give you 5 seconds to stop the boot (by pressing any key)

it will label OS *usually and OSX,

choose OS for Vista

choose OSX for Mac OSX



to upgrade to 10.5.3

boot into osx

place the drivers / 10.5.3 combo pkg update into the CD drive

-click and install the combo package update (it will ask you to reboot at the end DONT!)

-click the kernel package (install both kernels)

-now reboot

-hit the F8 key in the boot loader

-type update -v



The system will boot and possibly reboot ... then you will need to reinstall the ApplceBCM440xEthernet driver



reboot into osx

place your OSX drivers CD back in the drive and
go to system/library/extensions/IONetworkingFamily.kext/contents/plugins and drag AppleBCM440xEthernet.kext from the CD in there and replace


Open Applications->Utilities->terminal

(type the following)

sudo -s

*give password*

cd /system/library/extensions

rm extensions.mkext extensions.kextcache

chown -r root:wheel *.kext

kextcache -k /system/library/extensions

exit

Now reboot


Now everything should work fine again.



Last but not least the clock fix

boot into osx and unset the clock to update from apple automatically

then go into applications->utilities->terminal

1. Create new file /sbin/localtime-toggle with the following contents:

type vi /sbin/localtime-toggle

type i




type ESC

type :wq!



2. at the promple type:

chmod +x /sbin/localtime-toggle

3. Create new file :

at the prompt type
vi /System/Library/LaunchDaemons/org.osx86.localtime-toggle.plist

type i




type ESC

type :wq!



4. Reboot.

(set your clock to update automatically in windows)

Please send me errors you run into or suggestions on how to make this better

Friday, March 7, 2008

Simple Bulk rename : A Bash file rename script

often I have to rename a whole set of files that have not processed correctly, to do this I rename them with Bash, a simple script to do this is :

for f in *_3; do mv "$f" "${f/_3/_0}"; done

this renames all files ending in _3 to the same filename but now ending in _0



Tuesday, January 15, 2008

Freebsd PHP ports tree

So I have noticed that from time to time the /usr/ports/lang/php5-extensions versions get out of sync, to fix this its as simple as just removing the config (make rmconfig [only if you want to add more modules]) and rebuilding make install -DFORCE_PKG_REGISTER

Monday, November 19, 2007

Mythtv and Centos Gnome Autologin

After a bit of time playing with Media portal, which is GREAT, I have gone back to mythtv, which is not quite as intuitive, and setting up a recording is the worst ui experience, I only hope it gets better, but step one for me is to get the autologin working on Centos 5 the howto is below

Super simple way:
go to your mythtv user dir (~mythtv), make sure that the mythtv user has a valid shell:
vipw
in vipw "/mythtv"
if it has /sbin/nologin as the shell change it to /bin/bash or your favorite shell
next edit your /etc/gdm/custom.conf
add:
[daemon]
AutomaticLoginEnable=true
AutomaticLogin=mythtv
or
TimedLoginEnable=true
TimedLogin=mythtv
TimedLoginDelay=20
for a timer
then
edit ~mythtv/.xsession
add the lines:
#!/bin/bash
metacity &
i#xterm &
mythwelcome
chmod 755 ~mythtv/.xsession

Thursday, November 1, 2007

X11 Display on Vista with ssh

Need an X11 interface that works quickly and for free... install Cygwin with X11 and openssh.... then make sure your ssh on your remote host has X11forwarding=yes.

from your bash shell in cygwin type startx a x11 xterm will come up and type:

DISPLAY=localhost:0.0 ssh -Y remotehost


you set... then any remote x app will appear on your desktop

Tuesday, April 24, 2007

WAMP the solution for Vista

I needed to install PHP, PhpMyAdmin and MySql on Vista. This was taking forever, Apache 2.2 kept crashing, Pear wouldnt install ... so I looked up apache, mysql, php and windows and came across WAMP (Website). This is a GREAT package that save a ton of time. If you need to install Apache, Mysql, and PHP on a windows server, this is a must have package.

Friday, February 16, 2007

Stderr .... output to grep

When you use some program like spamassassin, you will notice that the debug information :

spamassassin -D --lint

goes to stderr ... so you cant use grep as straight forward as you want. Here is how to pipe it to grep in bash:

spamassassin --lint -D 2>&1 | grep ....

or spamassassin --lint -D > output.txt 2>&1

to pipe it to a file first.


Disk Hog on Unix

So it often comes to the time to find what is taking all the space on your hard drive and like many things in unix, this isnt so straight forward, du (disk usage) will tell you how to do this, here will show you your top ten offending directories:

du -ha / | sort -n -r | head -n 10

du -ha = disk usage in human readable format ( lowest to highest )
sort -n -r = reverse the order
head -n 10 = top ten

| = pipe the output to the next command


Tuesday, January 2, 2007

How to save a mapped drive password in XP

The manage network passwords utility in XP doesnt have an add password function so you have to go to the command prompt and use the following :

NET USE U: \\NETHOME\USERID /PERSISTENT:YES /SAVECRED

This will prompt you for the Username and password and store them.

Thursday, November 30, 2006

Simple Script to check if a process is running and if not restart it

I wrote this simple script to check if a process is running and if not restart it



#!/bin/sh
TEST=`pgrep $1`
if [ ! "$TEST" ]
then
$2
else
echo "$1 Alive "
fi
Usage:
check.sh
Example:
check.sh named /usr/local/sbin/named

Thursday, November 9, 2006

ddns for linux / unix in a simple cron job

Here is a script to update your dynamic dns for linux / unix...


edit a script:



vi /root/ddns.sh



put in ddns.sh :







chmod 755 ddns.sh



then store your crontab:



crontab -l > /root/crontab.new



edit the /root/crontab.new and add the line:



*/15 * * * * /root/ddns.sh &>/dev/null



then load the new crontab:



crontab /root/crontab.new



Your done! ddns is all ready to go!


Thursday, September 28, 2006

Monday, September 18, 2006

Pc hdTv3000 Firmware

Incase you want to know where hotplug firmware goes:

in fedora core 5 it goes in /lib/firmware/

Mythtv rc.local startup additions

My rc.local for mythtv on fc5

/usr/bin/mysqlcheck -u root -pxxxxx -r --all-databases
/sbin/modprobe lirc_mceusb2
/sbin/modprobe bt878
/sbin/modprobe dvb-bt8xx
/sbin/modprobe dst
amixer -c 0 sset Line,0 100%,100% cap mute
amixer -c 0 sset Capture,0 100%,100% cap mute
amixer sset 'IEC958 Playback AC97-SPSA' 0
mount -t cifs -o username=administrator,password=xxxx, //xxx/network /mnt/network
chmod -R 777 /dev/dvb/adapter0
chmod -R 777 /dev/dvb/adapter1