PVRs, file transfers, large hard disk drives and odd filesystems…

OK this is a somewhat minority interest post, but it might help someone who is trying to do the same tasks as I was, or experiences some or all of the same difficulties.

First some keywords for the search engines: PVR, ext2, ext3, linux, FAT16, FAT32, FAT16/32

OK first up: what was I trying to do:

• transfer files from my PVR (personal video recorder, a Humax Foxsat HDR) to my Windows 7 PC, for the purposes of archiving, and converting to compressed MP4 files using Handbrake, for playing on my tablet (a 1st generation Google Nexus 7)

Problems:

• Some files are too large to be transferred. Previously I was using a portable external drive of about 30GB size, which in actual fact was a repurposed iRiver MP3 player (iHP140) which I had long ago installed some 3rd party firmware (Rockbox) to extend its functionality, and subsequently stopped using when I got a smartphone that doubled as a portable music player.
• 
• This device could be mounted as a USB drive, and I believe it was formatted in FAT 16 or FAT32. This has a filesize limit of 4GB and some of the uncompressed video files (for exampla a movie longer than about 2 1/2 hours) would exceed this limit.
• The iRiver was getting a bit flaky and I suspect, recahing the end of its reliable natural life. Sometimes it would not be recognized, or files would randomly disappear, then reappear when it was powered down, rested, and remounted.

Initial solution seemed simple. External hard disks these days are larger, cheaper (and one hopes) more reliable. PC World had a good deal on Western Digital 1TB “My Passport” small external drives for £60. I snapped one up.

First problem encountered. The Western Digital HDD was formatted in NTFS. This was not recognized by my Humax PVR. Since there is no chance to alter the Humax, my options were limited. It recognizes either FAT 16/32, or linux EXT3. So if I was to continue my plan I would have to reformat the WD HDD to one of these. But which?

FAT 16/32 has the advantage of very wide compatibilty. Windows of all flavours would read it, and so would the Humax. But it is somewhat ill suited to the large size of modern HDDs. On Windows you can format a drive in FAT32 but with some severe limitations. The first is that maximum volume size if a meagre 32GB. Not very useful if your drive capacity is 1TB! Yyou can use one of many 3rd party partition manager programs to format a FAT32 drive with a larger volume size than this, but then you also come across other potential problems: cluster size, file allocation table size and performance. Now I don’t pretend to really understand the intricacies of this sort of stuff but some more info on this is here.

Anyway, I read enough to decide that maybe formatting to FAT16 or FAT32 was not likely to be the best use of my new 1TB drive, so I decided to investigate formatting in EXT3. Now what is EXT3? I knew that it was one of the commonly used filesystems in many Linux systems, but apart from that I was ignorant. I soon discovered that Windows doesn’t read this filesystem natively, and this was going to be a major problem in my original task of transferring files from the PVR to the Windows PC.

However a quick search revealed that there were a number of ways to allow Windows to access, at least on a read-only basis, EXT filesystems.  The EXT2fsd project seemed to be one of the one that came up most frequently in searches. However I was made a bit wary by the fact that it doesn’t seemed to have been updated since 2011, and also many webpages mentioned its risk of data corruption and other teething troubles. It sounded like a possible solution, but not perhaps ideal.

Finally I settled on another solution that sounded a little more stable and also seemed to allow writing a Windows system to a EXT3 system – Paragon Software EXTfs for Windows.

I downloaded this and installed it easily on my Windows PC. I then downloaded a ISO CD ROM image of a linux partition manager software called GParted. This worked very well and I was able to boot to Linux and using Gparted, formatted the Western Digital HDD to EXT3. I thought it might take a very long time on account of the large size of the drive, but actually it only took 11 minutes. Then I mounted it using the Paragon EXTfs software and it was recognized straight away. It appeared in Windows Explorer just like any other drive.

This is easy! I did a quick test file transfer of a small file from Windows to WD without problems. Next up I deleted all the files to start from a blank slate and connected the WD to my Humax. Recognized! Success! I straight away transferred one of the files that had been too large for my old iRiver. No problems there. Connected back up to the Windows PC and transferred said file to the laptop. All my problems are solved. (Or so I thought).

The Sting In The Tail

I was about to have some annual leave, and one of my ideas was to catch up on some TV that I had been missing recently. I transferred a large amount of video and audio files to the WD drive. I checked them on the Windows PC using the Paragon extfs driver. Then I relocated to where I was spending my leave (where there is another Windows PC and a different PVR, this time a Humax Freeview FOX-T2 HDR – note this is different model from the PVR I have at home which is a Freesat one.)

First thing I wanted to do was watch one of the files I had transported on the WD HDD, but on my Nexus tablet. I installed Paragon extfs on the PC in my new location and transferred the file there. Then I transferred to my Nexus without problems.

Now came my big mistake. I discovered that the 2nd Humax PVR (the Freeview one) had a recording that I wanted to keep. I thought I could simply connect the newly EXT3 formatted external drive to it, copy the file onto it, then onto the Windows PC to shrink using Handbrake, then onto my Nexus or just keep on the external drive until I got back home in a week. After all I wasn’t pushed for space on this 1TB drive.

However, when I connected it to the Freeview PVR and browsed to the drive, I was a bit surprised to find it appeared “empty” instead of the dozen or so files that I had just looked at a few minutes before. I thought it was a bit odd, but continued to copy the file from the Freeview PVR onto the WD HDD.

When I connected it to the Windows PC and viewed it using Paragon extfs I was alarmed to see that now all those files that I thought were safely stored on the external drive, were GONE! In their place was the newly transferred file from the Freeview PVR, and a small collection of files that I had previously transferred from my other PVR a couple of days ago, but subsequently erased (or so I thought).

It soon dawned on me that somehow, by connecting the external EXT3 formatted drive to the second PVR I had caused the old files to reappear and newer ones to “disappear”. Don’t ask me the technicalities of what had happened. Some linux guru might be able to explain. However I suspected that they might still be accessible and recoverable…. somehow. I had only written one time to the drive (the transfer from the second PVR), so I thought there was a fair chance most of the data would sti8ll be intact.

I searched for file recovery/undelete software. This needed to work on EXT partitions, but run from Windows, so this narrowed it down a bit. To cut a long story short, I tried a few. DiskInternals have a piece of software called Linux Reader which I thought might fit the bill. However after installation it seemed to take an exceedingly long time to read the drives. Eventually I gave up and searched again. Next I tried EaseUS Data recovery Wizard. This actually did work in that it did eventually (after a good many hours in the background) read and index all the files on the WD external drive, including all the ones that had “disappeared” and which I wanted to recover. BUT, when I tried to recover them I found to my dismay that the free trial version only allowed you to view and search but to actually recover the files you had to buy the “pro” version for $80.

Finally, I tried a free piece of software from RTT (R-Tools Technology, @RTT-inc on Twitter) called R-Linux. This looked like it was going to take an unrealistic time to scan my 1TB drive (the “time remaining” indicator went up to 5 days!) but it actually completed the scan in about an hour. After that I was successfully able to recover (and transfer intact to a Windows folder) most of the files I had lost. A few were incomplete, probably as a result of some disk writing on the second PVR but I was able to get most of what I wanted and it didn’t take too long.

So, the moral of the story is that ext3 formatted drives still don’t play perfectly with Windows systems, but probably using Paragon’s extfs for Windows software is your best bet. Be careful and you should be OK. Don’t mix and match the systems you connect the drive to any more than you have too, and back up frequently (always a good idea in any computer related activity). Finally if you get into a pickle, RTT’s R-Linux is a saviour.

Switching to Freesat

One of my New Year’s resolutions was to extricate myself from the clutches of Rupert Murdoch and ditch Sky. One of the reasons for this was I want to move to having HD input. My long term strategy is to get a bigger TV but I can’t do it all at once, and it makes sense to upgrade the source first and the display later.

So, my previous set up was: Standard Sky, via old-style UK TiVo (Series 1) box, into a 28″ Phillips LCD TV. The Sky is the most basic package. Even with this I don’t really get the time to watch much TV. So my goal is less but better quality TV.

The first part of my strategy was achieved by buying a PS3. This means I have Blu-Ray option for HD movies. I have now executed part two of the strategy and taken delivery of a Humax Foxsat HDR (500GB model, £280).

Set Up

This was incredibly easy. I already had a satellite dish for my previous Sky, and I had two inputs (the previous owners of my house had Sky multiroom). After plugging in the two satellite feeds and powering up, it was a few simple setup screens for language preference, postcode etc, and then a short scan for channels and that was it.

Picture Quality

To me it looks pretty good. Even the standard def channels seem clear and I realise now that my prevous picture was likely being degraded by having SD going through the TiVo (some compression). The HD channels (currently BBC1 HD, BBC HD, ITV1 HD, and from April Ch4 HD) look very nice, especially compared to what I was used to. Of course with my relatively small TV I don’t think I am getting the full benefit of the HD, but as I explained earlier this is a future proofing strategy and hopefully I will eventually upgrade to a larger TV.

User Interface/Programme Guide

The Freesat system has a 7-day programme guide. The UI is neat and tidy and professional looking. I don’t know if it is specific to the Humax box or is standard for Freesat in general. I suspect the former. It is pretty easy to change channels, either direct type channel number, use up/down or via guide. The guide is a little cramped. You have a list of channels down the left colums, and programmes on right. There is only enough room for about 2 hours worth horizontally so it is a little clunky to scan forwards in time to see what is on later in the evening. You can see about 7 channels at once vertically. The extra space at the top is used to display programme info. Overall roughly half the screen real estate is used to display actual guide data. It looks neat and clear, but you have to scroll a lot.

There is an alternate view called “List” that instead lets you focus on one channel at a time. It lists schedule for one channel vertically and you can see 7 programmes at a time which is a useful amount. You can scroll up and down one programme at a time, a page at a time, or one full day at at time. These options mean you can quite quickly see what you are looking for.

The “Schedule” screen lists the programmes you have lined up to record. Seven programmes are listed vertically and you can page up/down with the channel up/down buttons.

Finally a “Search” page is one of the ways to find programmes to record. This is reasonably functional, but is a long long way behind Tivo search. You can do a basic keyword/title search which will probably find what you are looking for. Once found you can click and choose to record individual instances of a programmes, or a whole series. The concept of setting up a season pass (as on Tivo) is not quite the same and you can easily list and edit all your season passes. Also you cannot set up a search for programmes that are not yet in the guide and then relax in the knowledge that they will be picked up later and recorded. This, combined with similar keyword/title/actor Wishlists was a major advantage of Tivo, and one I greatly miss.

On Tivo for example you could set up a wish list for “cricket” and all programmes that had that in the title or description would automatically be recorded as and when they came up. Or one for “Hopkins” (the actor) or “french” if you were interested in learning the language (although you get a lot of French language films and programmes with French settings or actors as well…) With Humax Freesat your searches and recording options are a bit more limited.

Your store of recorded programmes is in a separate menu that also can show other files (recorded radio programmes are separately listed – quite useful, and music and photos which you can upload from USB are also available). The recorded programmes can be listed in alphabetical order by date, and the box automatically groups episodes from the same programme in separate folders, again a useful feature.

One final point about the guide. When you call it up, it refreshes with latest data. This is good but slow and it means the currently playing programme is silenced for up to a minute. There is also a slight delay before you can start navigating the guide which can be annoying. These “bugs” are rumoured to be on the list to be fixed in the next software release.

Recording

Apart from searching through the programme guide and picking future programmes to record, you can also record “instantly” while watching a programme by pressing the record button. The big drawback (and really unforgivable flaw in my opinion) is that even if the live buffer contains the start of the current programme, when you press record you can only capture from that moment onwards. What is the point of the buffer? I hear that this may be fixed in a future software release, but this really is a fundemental flaw. Another annoying thing is that the live buffer (which is otherwise a very useful 2 hours) automatically resets not only when you change channel (understandable) but also whenever you view a previously recorded programme, or even view the guide. A bit annoying to say the least.

Playback/Navigation

Playback is pretty standard for PVRs. Pressing play displays a time bar across the bottom. You can fast forward and back at variable speeds (2x – 64x) with multiple presses of the FF/RWD buttons. The jump back and forward buttons are also definable to an extent (back: 7, 15, 30s; forward: 30, 60, 120, 240s) This gives a good way of skipping easily through ad breaks, and of replaying incidents (e.g. goals?) you might just miss. If you need to navigate more quickly through a long programme the left/right cursor keys drag a cursor through the timeline very quickly. Also there is a bookmark function. You can set multiple bookmarks throughout a recorded programme and then jump through them with left/right keys. Each bookmark gets a image-captured thumbnail to remind you of what it is. Overall the navigation features are very good.

Other Features

• Channels can be locked via a PIN, or restricted to certain times (using rating data so that 15 certificate programmes for example will only show after 8pm)
• Subtitles and audio description are displayed where available.
• Sound is reproduced with Dolby Digital where available.
• You can set an on-off time for the machine.
• Power saving mode. Auto-switch off after period of inactivity (does not affect recordings).
• IP programming: If you connect to you broadband via ethernet (e.g. using a Homeplug network) you can watch BBC iPlayer and ITV Player as channels on the programme guide. I haven’t set this up so can’t comment on this implementation. I use my PS3 to watch iPlayer/4OD/ITV player.

Summary

This is a very good machine, though not perfect. It does most of the core features required of a PVR and does them well. It is marred by a couple of software design flaws (the handling of the live buffer is poor, and the responsiveness of the guide is a slow). There are idiosyncracies of the UI but only the most picky people would find them unacceptable. In most cases it is just a matter of becoming used to how to achieve a task with this particular machine. The search/season pass functionality is adequate, but not a patch on how TiVo did it. However navigation and playback options are particularly good.

500GB is plenty for SD recording, however if you record and store a lot of HD content you might find it a little tight. There is a new 1TB model coming out for the horders amongst you.

For many people the overall choice of whether to go with Freesat will be determined by which channels they particularly want to watch. (I have made a separate post in more detail about the channels). There is no doubt that the range of channels on Freesat is quite limited. However rememebr this is not a subscription service. If like me you find that 80% of your viewing is BBC and the rest is Ch4/more4/Film4 then this is a good deal. I used to have the basic Sky package (Variety & Knowledge pack) so I lose a few channels that I liked particularly FX (The Walking Dead, True Blood), SyFy and the Discovery channels (Mythbusters and a few interesting shows from time to time) and occasionally Dave (where the new series of Red Dwarf will eventually come). Also the upcoming new Sky Atlantic channel looks interesting. [FULL CHANNEL LIST]

However I have come to realise that I couldn’t even keep up with this degree of choice and I think I can live with the smaller range. The money I will save will let me buy boxed sets of those series I really wish to see if they don’t come to BBC/Ch4 eventually.

(Video courtesy of Decipher’s YouTube channel)

Overall: 4/5 (will get an extra 0.5pt if they fix the flaws with a software update)

2011 New Year’s Resolutions Update

How am I doing with my NY resolutions?

Around the turn of the year I tweeted my New Year’s Resolutions. Briefly, they were

1. Quit Sky (the satellite TV provider, and part of the Murdoch evil empire)
2. Try to not leave so many videogames unfinished
3. Try and read a bit more (a general one that features every year)

Quit Sky

Well on the Sky front, I haven’t really done anything. My reason (over and above a general dislike for Murdoch) is the increasing cost (currently £20.50 pcm), a realisation that I actually only occasionally watch TV, and most of that is on the standard BBC channels (occasionally the digital channels), and a desire to upgrade to an HD source (I certainly won’t be paying the premium to get Sky HD).

My masterplan was to switch to Freesat HD. I already have the satellite dish and although the selection of channels on Freesat compared with even the basic Sky package is much smaller, it mostly covers the things I like to watch. The only things I would miss are very occasional shows on FX and maybe SyFy or Sky1, but this is rare. I wanted to get a Freesat HD recorder and the Humax 500GB model was the one I had my eye on. It costs £300 but this would pay it self off by just over a year of Sky subscriptions. I would lose a few channels, and gain BBC HD, BBC 1 HD, ITV HD and any other future channels which launch in HD on Freesat (hopefully). So, I will probably execute this plan soon.

The imminent launch of Sky Atlantic almost put a spanner in the works of my plan. Dustin Hoffman is quite persuasive, and HBO is a purveyor many quality shows. However having considered it carefully, I know I don’t have enough time to watch all those shows anyway so I should just forget about it. If there is a series on there that I really must have, I’ll just have to buy the box set DVD/BD.

—

Videogame completion

This one is not looking so good. Here is my list of games currently uncompleted.

• Super Mario Galaxy 2 (Wii)  – enjoying it, but still only about 1/5 of way through.
• Resident Evil 4 (Wii) – Not played it for ages, but I never actually got right to the end. I’m sure I could polish it off in a weekend though if I really tried.
• Monkey Island 2 (SE) (iPhone) – only recently purchased, after I actually completed MI 1, but I have only got a little way into it, so this will take some time.
• Bioshock (PC) – Bought ages ago. Really enjoyed but got distracted about half way through. Will be quite a struggle to finish I think.
• Bioshock 2 (PS3) – Recently bought when I got the PS3 as it was cheap. A very good game, but I’ve only scratched the surface…
• Blade Runner (PC) – This is a classic point-and-click adventure from 1997, which I bought off Ebay after it was featured in Edge Magazine. It really is a game ahead of it’s time in terms of atmosphere and style.

The real problem is when you consider the next list: games I really am tempted to buy

• Mass Effect 2 (PS3) – This has just been released and I don’t know how long I will hold out. I downloaded the demo and it’s really, really good.
• F1 2010 (PS3) – This is actually one of the reasons I was tempted into getting a PS3 in the first place. There’s only a couple of months to the start of the new F1 season and so I think this will be bought pretty soon….
• Uncharted 2 (PS3) – Generally agreed to be one of the best PS3 games. Ever.
• Red Dead Redemption (PS) – Another really well reviewed game which is now just old enough to be pretty cheap too.
• Resident Evil 5 Move edition (PS3) – OK, I really should at least complete RE:4 before I buy this.
• Deus Ex: Human Revolution (PS3) – this is not out yet, but it’s not long. A sequel to a classic game which looks good and which fans of Blade Runner-esque settings will find hard to resist.
• Beyond Good & Evil HD (re-make) (PS3) BG&E is an old game that was released on GameCube and PS2 and is another contender for “best Zelda game that is not actually a Zelda game” (other contenders for this title include Okami). Now it’s being spruced up with fancy new graphics and re-released for PS3…
• Shadow Of The Colossus & Ico (remake) (PS3) Underrated, but much acclaimed PS2 games, now coming to PS3 with spiffy HD visuals. 91% and 90% respectively on Metacritic.
• Portal 2 (PS3) – Luckily this is not due for some time. But Valve’s new found love affair with Sony means this might be the definitive version, and Jonathan Coulton has recently confirmed that he will again be providing a song. (Could it ever live up to the classic that was Still Alive)

—

Read more

Well, so far I have read precisely “zero” books. On the plus side I have listened to 1 audiobook and am about halfway though another so that’s something. The one I completed was a disappointment however. (The Michael Moorcock Doctor Who title “Coming Of The Terraphiles”).

To do list for books:

• The Angel’s Game (sequel to Shadow of the Wind)
• Our Kind Of Traitor (Le Carré, audiobook)
• Operation Mincemeat (in progress, audiobook)
• The God Delusion (Dawkins)

Wish me luck!