Updated Instructions This topic is for all those people out there who have realised that Zap2It are no longer supplying free data via there Data Direct facility. Basically we have written a new tool that is better, more flexible and supports the Yahoo free data download as well as the new Schedules Direct paid system. Please post questions, issues, suggestions as separate topics please.
Finding Bugs
Please report bugs in BugIt.
Downloading
Try it out at http://accounts.digiguide.com/installerfiles/signedsetups/dglg10-setup.exe
Installation
You ought to install it into a subfolder within the DigiGuide installation folder. When run it will ask you the location of your DigiGuide installation so it can find out where to put schedules and where your profile is located.
Usage
If you are a user who has used XMLTV or DD2DG before then you need to cleanup your copy of DigiGuide first otherwise you will get all the channels doubled-up.
* run DigiGuide
* choose "Add/Remove Channels" (Tools menu)
* click "Show advanced options"
* select the very last channel in the left hand "lineup" list
* repeatedly click "Delete channel" until it is disabled and there are no 'independant' channels left
* run Listings Grabber
The 'first time' process for using the grabber is to:
* run it
* click "Add New" and go through the steps to add a 'grabber' for your zipcode/provider/device
* select the new grabber and click "Run Selected" Using the command line
You can also run the tool from the command line (to setup a Windows Scheduler Task for example) with the following options to just process the data without using the GUI:
DGLG.exe -n -f="C:\Program Files\DigiGuide TV Guide\Listings Grabber\config-files\my_usa_grabber.dat" -n ... means dont show the GUI, omit this option to display a GUI
-f="<path to a grabber.dat file>" ... specifies the path to the grabber dat file you wish to process, you can have multiple "-f" parameters to launch and process multiple grabbers Alternatively, just click the new "Schedule Run" to get a Windows Task Schedule created ... it popups a dialog which you can use to copy the 'path'.
Customisation/Control
Within "config-files" there are three folders:
* channel-maps : allows you to specify custom names, logo paths, epgs and channel categories for any channel. For example, "SCIFI" could become "My Sci Fi Channel" with EPG "TV0010".
* category-maps : allows you to specify the category and keywords used for any source genres that get imported. For example, "Special Interest/Science" could get the "Scientific Documentary" category.
* keyword-additions : allows you to add further keywords to a programme based on its category and/or existing keywords. For example, "Film" category programmes could get additional "Movies", "Films", "Movie" keywords.
Within each of these folders is a "samples" folder that contains a 'default' file which could get overwritten by any new installations. When DGLG runs, it copies the defaults out of the 'samples' folder into its main folder if it doesnt already exist. You can therefore edit the files in the root of the 3 config-files listed above happily without fear of them being overwritten. The naming convention is as follows:
DGLG will look for a file "<yourgrabbername>.channel-map.dat", if this cannot be found it will look for a file "<grabbertype>.channel-map.dat", if this cannot be found it will use the "default.category-map.dat" file that should always exist. (Replace the "channel-map.dat" file with "category-map.dat" and "keyword-additions.dat" where appropriate.) The "<yourgrabbername>" is the name you see in the list like "My DishTV provider grabber". The "<grabbertype>" will be the type of the grabber, for Yahoo this will be "USA Yahoo Go TV", for the Schedules Direct grabber it will be (when its written) "USA Schedules Direct". With this capability, you can create custom files for each specific grabber, or one for any grabber for a type, or just use the defaults which will be used for all grabbers for any type. This should give maximum flexibility. Extended Episode Details
When enabled with the "Use enhanced episode info", Listings Grabber will download and use the official extended episode guide information to add series/episode numbers and correct episode names. As this is primarily from our database we may be behind on USA shows so you can either tell us which shows are missing numbers and we will add them, or you can create your own "epguide.dat" file to enhance the details yourself. A sample file with documentation is in the "%APP%\epguide\samples\" folder. Extended Information From IMDb
If you wish to extend the program information using some additional IMDb information, you will need to download some files from IMDb by clicking the "Get IMDb Data" button. This may take a while as the files are quite large. Once downloaded, the Grabber will automatically import the data into an internal database for speed and use it in programme schedules if ticked on the interface. From http://www.imdb.com/interfaces/.
Please refer to the copyright/license information listed in each file for instructions on allowed usage. The data is NOT FREE although it may be used for free in specific circumstances. Channel Logos
Create a "channel-images" folder within your installation folder and place your image files as "<callsign>.png". As a starting point, download our latest list of images from http://i2.digiguide.com/usalogos.zip
Changes In Build 24
MSN and MSN Canada Grabbers: Fixed due to change in MSN Site format: http://bugit.digiguide.com/list_viewitem.asp?bugid=1743
Zap2It Grabber: Fixed change in Zap2It site format for channel names: http://forums.digiguide.com/topic.asp?id=25365&subject=Not+getting+new+listings+%2D+using+DGLG+with+Yahoo
Yahoo Grabber: Fixed change in receipt of channel list due to response headers being included: http://forums.digiguide.com/topic.asp?id=25365&subject=Not+getting+new+listings+%2D+using+DGLG+with+Yahoo
Yahoo Grabber: Now reports if schedules.zip fails to decompress (probably due to it being a server error text file being saved as a zip file)Changes In Build 23
Built for new DigiGuide v8.4 user profileChanges In Build 22
Internal buildChanges In Build 21
Fixed Zap2It grabber to handle changes to their site
Built for new DigiGuide v8.4 user profile and settings.dat filenameChanges In Build 20
Built for new DigiGuide v8.4 user profileChanges In Build 19
Fixed issue with old listings not being purged according to setting valueChanges In Build 18
Built for new DigiGuide v8.4 user profileChanges In Build 17
Fixed bug where datetime stamp of import was not being saved
Fixed bug in schedule creation: ]http://forums.digiguide.com/topic.asp?id=22493Changes In Build 16
MSN Canada/USA Grabber: Fixed a bug where old schedule files where not redownloaded, so perhaps if they were empty when downloaded you would get gaps in your schedules
Scripts: Fixed bug where an invalid or empty start time would produce a script error (which was wrongly named anyway): ]http://forums.digiguide.com/topic.asp?whichpage=99999&id=22461#end
MSN Canada Grabber: Fixed bug where postal code field was not big enoughChanges In Build 15
Changed interface to be much cleaner and simpler, e.g. options like using episode info and IMDb are always on, but you can turn them off from "Options", also added more instruction info for first time users
Added MSN Grabber for Canada
MSN Grabber: Fixed various issues with script errors when strange characters appeared in the description
MSN Grabber: Fixed a bug where the episode, cast, directors where not converted from utf8 to Latin, fixed a bug in JSON data conversion where the \\" was not escaped properly
XMLTV grabber: fixed timezone issue, where the offset from GMT was not parsed
Zap2It Web Site Grabber: Fixed bug with duff data arriving on channels (possibly due to Zap2It having no data available for a channel) and therefore into DigiGuide causing random gaps to appear
Lots of additions made to sample category maps, so delete the category maps you have not modified from the "category-maps" folder and let DGLG copy over the new ones from the Samples folder when it next starts
Added ability to stop the parsing of large XML files, so you can stop the episode guide parsing in the middle
Fixed description download and categorization based on keywords
Fixed bug with wrong information being passed as grabber name/type for JavaScript grabbers, so that channel/category-maps couldnt be customised
Now corrects case of any qualifiers added (use Uppercase.txt and Lowercase.txt to override the auto-capitalisation)
Modified so that the "Run Time" qualifier is only added if it is different to the actual programme duration
Added ability for channel map to provide a URL parameter: http://forums.digiguide.com/topic.asp?id=22302
Modified synd ep num conversion to work on programmes without episode titles: http://forums.digiguide.com/topic.asp?id=22309 Changes In Build 14
Fixed an unforseen bug if stopping a 'run' and then clicking "Add New" would result in a bogus error messageChanges In Build 13
Added more category maps to the "default.category-map.dat" file in "Samples", if you havent edited the default, then copy this one over
Added version information to all DLLs with new exported function
Added logging of DLL loading
Added version number to dialog title bar
Added a Logging option to output verbose log output (this is not saved, but will output more information into the log file for debugging)
Added a slight change to the Add New dialog tree control to lower colours used in tree images (possibly to fix errant crashes)
Added optional logging of 'unknown' or 'empty' categories .. enable from new Logging tab on Options
MSN Web Site Grabber: Fixed potential issue with MSN grabber and bad downloaded description data
Zap2It Web Site Grabber: Added check for no data being added, when Zap2It delete someones account or the data is wrong then an error page occurs and no channels are added.
Zap2It Web Site Grabber: Fixed bug with extraction of programme genres on programmes that have 'sponsored' information on them
Fixed crash when clicking Add New and the users locale is a country that isnt supported by any grabber
Set global default of unknown categories to Entertainment for all grabbers
Made slight change to ignore any category map changes to programme categories if it has already been fixed by the episode guide
Fixed bug where converting syndicated episode info would always occur regardless of the option to not do it: http://forums.digiguide.com/topic.asp?id=22268 Changes In Build 12
Added new MSN Web Site grabber
Added a General Options tab, so various settings can be easily modified
Added an IMDb Options tab, so setting what gets used is much easier
Added new setCursor method to system object to be able to set wait pointers from within JavaScript UI functions
Set default category to "Entertainment" for all programs on Zap2It Web Site Grabber to avoid "unknown" categories from appearing
Fixed issue where intellidownload would download more than 14 days if it was last downloaded more than 14 days ago
Fixed issue where syndicated episode number was not being cleared out between schedules so programmes would inherit previous programmes syn number even when it shouldnt have one
Fixed issue where setting IMDb options would get ignored and the defaults would always get used
Fixed issue where you couldnt 'remove' cast/actors as it always set it back to the default
Fixed crash bug on clicking "Add New" if no Grabber DLLs were present
Added automatic normalising of some qualifiers so all grabbers user or system created will convert things like "HD" to "High Definition" all the time
Fixed bug where the yahoo grabber wasn't correctly using the run time field from the data: http://bugit.digiguide.com/list_viewitem.asp?bugid=1401 Changes In Build 11
Added new MSN Web Site grabber
Added a General Options tab, so various settings can be easily modified
Added an IMDb Options tab, so setting what gets used is much easier
Added new setCursor method to system object to be able to set wait pointers from within JavaScript UI functions
Fixed issue where intellidownload would download more than 14 days if it was last downloaded more than 14 days ago
Fixed issue where syndicated episode number was not being cleared out between schedules so programmes would inherit previous programmes syn number even when it shouldnt have one
Fixed issue where setting IMDb options would get ignored and the defaults would always get used
Fixed issue where you couldnt 'remove' cast/actors as it always set it back to the default
Fixed crash bug on clicking "Add New" if no Grabber DLLs were present
Added automatic normalising of some qualifiers so all grabbers user or system created will convert things like "HD" to "High Definition" all the time
Fixed bug where the yahoo grabber wasn't correctly using the run time field from the data: http://bugit.digiguide.com/list_viewitem.asp?bugid=1401 Changes In Build 10
It works now!!Changes In Build 9
Changed JavaScript engine to garbagecollect more frequently thus improving memory usage by a large amount
Improved calculation of when a script needs to be auto-updated
Improved Zap2It Web Site Grabber to conserve memory usage and avoid excessive downloading of files
Added button on dialog to download the IMDb data directly
Modified HTTPGet/Post to set filetime to the same as the "last modified" date of the header if any
Modified to calculate durations of a schedule list automatically if any programme has no duration
Added ability for "shellExecute" to 'wait' for process to finish with an additional flag at the end
Added JavaScript documentation for the decompressFile: http://www.digiguide.com/dev/dglg/javascript-grabber/decompressFile.shtml
Added messageBox JavaScript API: http://www.digiguide.com/dev/dglg/javascript-grabber/messageBox.shtml
Added JavaScript documentation for the HTMLWindow: http://www.digiguide.com/dev/dglg/javascript-grabber/HTMLWindow.shtml
Added JavaScript documentation for the HTMLParser: http://www.digiguide.com/dev/dglg/javascript-grabber/HTMLParser.shtml
Modified HTMLParser object to pass a Parameters object with properties for each parameter known so you can access a parameter directly like "alert( params.href );". If a parameter exists more than once then the property is an array of values for that parameter
Removed CustomPrograme.grabberName from JS API as it's not needed. Any scripts using it will need to be altered.Changes In Build 8
Added new Zap2it Web Site parser
Added ability for javascript grabbers to be automatically updated: http://www.digiguide.com/dev/dglg/javascript-grabber/Grabber.autoUpdateURL.shtml
Added new HTMLParser object to make parsing web pages much easier
Added decompressFile function to decompress a gzip file path given and return the uncompressed string value
Fixed XMLHTTPRequest post data handling
Added optional default parameter for grabber.getOption
Added double click on a grabber to run it (from the main window).
Given default XMLTV grabber name "my ... grabber" a nicer default
Added new 'tree' view for adding grabbers, separated by country and defaulting to your current OS country. Also made dialog sizeable and remember its size/position
Made XMLTV grabber names a bit nicer
Fixed issue where logos were not being used on non-Yahoo grabbers
Updated Radio Times JavaScript grabber to use new MultiHTTPRequest object
Added JavaScript include file for some handy JavaScript functions and objects (this will get overwritten, so dont modify it): #include "HTTP Helpers.jsh"
Added JavaScript API to grab multiple files at the same time: MultiHTTPRequest. See http://www.digiguide.com/dev/dglg/javascript-grabber/MultiHTTPRequest.shtml Changes In Build 7
Fixed bug with Original Air Date: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1092
Fixed bug where file date time's of downloaded files was incorrectly comparing against a localised date when it should compare against a UTC value of the date: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1372
Fixed bug where channel category was not being used on schedules that had a category already when it ought to be used on all schedules to ensure the 'correct' DG approved category is used: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1362
Added loading of category map to output
Added JavaScript Grabber Engine: See ]http://forums.digiguide.com/topic.asp?id=22015
Fixed bug where it was quite working right when invoked from the command line: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1360
Added CastToISO_8859_1 function to cast back from utf8 char
Added XMLTV grabber
Fixed bug where running a grabber from the command line didn't also enable/disable buttons correctly
Fixed bug where passing files on the command line wouldn't execute them: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1357
Added double-click to go next on the lists in the wizard for the Yahoo grabber.
Fixed bug where a SD wizard page had the wrong captyion: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1343
Fixed bug where the option to rename "close captioned" wasn't working correctly: ]http://bugit.digiguide.com/list_viewitem.asp?bugid=1345
Added some checks to validity of the 'programs' node: ]http://forums.digiguide.com/topic.asp?id=21835&whichpage=2&pagesize=15Changes In Build 6
More debug output for decompressing problems, also now writes the datetime of the lastwritten to the same as the zip as soon as its written
Fixed bug: http://bugit.digiguide.com/list_viewitem.asp?bugid=1338
Added options to remove the "Run Time" and "Syndicated Episode Number" qualifiers:
<option name="add-run-time-qualifier">0</option>
<option name="add-syndicated-episode-number-qualifier">0</option>
Added option to attempt to convert syndicated episode number into a series/episode number that can be added to the episode name. It only does this if there are no series/episode numbers already done. This is disabled by default so you need to add an option to enable it:
<option name="convert-syndicated-episode-to-numbers">1</option> Changes In Build 5
* Added a Schedules Direct grabber
* Added a button to create a Windows Task Schedule task for the selected grabbers
* Added support for custom episode guides, see sample in the "%APP%\epguide\samples\" folder for example and instructions
* Added setting option to rename the "close captioned" qualifier to something else like:
<option name="close-caption-name-override">closed-captioned</option>
* Added setting option to remove original air date information from qualifiers:
<option name="add-original-air-date-qualifier">0</option>
* Fixed bug http://bugit.digiguide.com/list_viewitem.asp?bugid=1067: You will need to force a "Reload of channels" to get it to change
* Fixed bug where special episodes would be converted wrongly into Series 65535 from the epguide
* Fixed potential problem where if DigiGuide took a long time to load up the channel information then you would get "could not find channel id" errors until you re-ran the grabberChanges In Build 4
* Added checking for availability of a new version into top of log output
* Added auto-download of epguide.dat file (containing enhanced episode details)
* Added ability to force the absorbtion of all channels
* Added support for absorbing keyword additions via an xml (.dat) file. This file follows the same naming system as category and channel map files.
Fixed crash bug when generating/upgrading an IMDb database
* Added progress bar for various long actions like: downloading data, importing IMDb, extracting episode data, processing data and generating schedules
* Added an application icon Changes In Build 3
* Moved all 'sample' data files to "samples" subfolders in their respective "config-files" folders. DGLG will copy the necessary default ones to the main folder if they do not exist and then it will not overwrite them so you can happily edit them if you wish, but can always refer to updated ones in 'samples' if necessary
* Added support for 'channel maps'. This allows you to customise the channel name, logo, epg and category of any channels. There is an example file in "%APP%\config-files\channel-maps". The naming follows the same as the category maps so you can have specify channel maps for grabbers, grabber types or just edit the global one.
* Added basic Options dialog
* Added an extra page in the yahoo config wizard so you can view the channels for the provider: http://bugit.digiguide.com/list_viewitem.asp?bugid=1327
* Hopefully fixed time zone issues: http://bugit.digiguide.com/list_viewitem.asp?bugid=1330
* Improved several log output messages and add a few more. Changes In Build 2
* Added support for setting channel logos. Create a "channel-images" folder within your installation folder and place all your images as "<callsign>.png". Download our latest list of images from http://i2.digiguide.com/usalogos.zip
* Added support to use IMDb data, read the IMDb.txt file within the IMDb folder for details on where to get the data
* Added better creation of channels within DG. Now DGLG will generate a new DGC file for any new channels you have and will automatically tell DG about it. So now there is no extra step to tell DG about new channels.
* Added saving of settings and options on dialog for choosing to include imdb/episode info
* Added better timeouts for downloading data, you can modify the defaults (in seconds) by adding the following lines into the "%APP%/config-files/settings/settings.dat" file:
<option name="connection-timeout" value="30" />
<option name="download-timeout" value="600" /> | 
