downmodis module

Module to download MODIS HDF files from NASA repository. It supports both FTP and HTTP repositories

Classes:

• modisHtmlParser
• downModis

Functions:

• urljoin()
• getNewerVersion()
• str2date()

class pymodis.downmodis.downModis(destinationFolder, password=None, user='anonymous', url='http://e4ftl01.cr.usgs.gov', tiles=None, path='MOLT', product='MOD11A1.005', today=None, enddate=None, delta=10, jpg=False, debug=False, timeout=30)

A class to download MODIS data from NASA FTP or HTTP repositories

Parameters:

• destinationFolder (str) – where the files will be stored
• password (str) – the password, it should be your email address to connect to a FTP server. Do not use this variable if the server is an HTTP server
• user (str) – the user name, by default ‘anonymous’, used to connect to an FTP server. Do not use this variable if the server is an HTTP server
• url (str) – the base url from where to download the MODIS data, it can be FTP or HTTP but it has to start with ‘ftp://‘ or ‘http://‘
• path (str) – the directory where the data that you want to download are stored on the FTP server. For HTTP requests, this is the part of the url between the ‘url’ parameter and the ‘product’ parameter.
• product (str) – the code of the product to download, the code should be idential to the one of the url
• tiles (str) – a set of tiles to be downloaded, None == all tiles. This can be passed as a string of tileIDs separated by commas, or as a list of individual tileIDs
• today (str) – the day to start downloading; in order to pass a date different from today use the format YYYY-MM-DD
• enddate (str) – the day to end downloading; in order to pass a date use the format YYYY-MM-DD. This day must be before the ‘today’ parameter. Downloading happens in reverse order (currently)
• delta (int) – timelag i.e. the number of days starting from today backwards. Will be overwritten if ‘enddate’ is specifed during instantiation
• jpeg (bool) – set to True if you want to download the JPG overview file in addition to the HDF
• debug (bool) – set to True if you want to obtain debug information
• timeout (int) – Timeout value for HTTP server (seconds)

checkDataExist(listNewFile, move=False)

Check if a file already exists in the local download directory

Parameters:

• listNewFile (list) – list of all files, returned by getFilesList function
• move (bool) – it is useful to know if a function is called from download or move function

Returns:

list of files to download

checkFile(filHdf)

Check by using GDAL to be sure that the download went ok

Parameters:filHdf (str) – name of the HDF file to check Returns:0 if file is correct, 1 for error

closeFTP()

Close ftp connection and close the file list document

closeFilelist()

Function to close the file list of where the files are downloaded

connect(ncon=20)

Connect to the server and fill the dirData variable

Parameters:ncon (int) – maximum number of attempts to connect to the HTTP server before failing

dayDownload(day, listFilesDown)

Downloads tiles for the selected day

Parameters:

• day (str) – the day in format YYYY.MM.DD
• listFilesDown (list) – list of the files to download, returned by checkDataExist function

debugDays()

This function is useful to debug the number of days

debugLog()

Function to create the debug file

Returns:a Logger object to use to write debug info

debugMaps()

Prints the files to download to the debug stream

downloadFile(filDown, filHdf, day)

Download a single file

Parameters:

• filDown (str) – name of the file to download
• filHdf (str) – name of the file to write to
• day (str) – the day in format YYYY.MM.DD

downloadsAllDay(clean=False, allDays=False)

Download all requested days

Parameters:

• clean (bool) – if True remove the empty files, they could have some problems in the previous download
• allDays (bool) – download all passable days

getAllDays()

Return a list of all days

getFilesList(day=None)

Returns a list of files to download. HDF and XML files are downloaded by default. JPG files will be downloaded if self.jpeg == True.

Parameters:day (str) – the date of data in format YYYY.MM.DD Returns:a list of files to download for the day

getListDays()

Return a list of all selected days

removeEmptyFiles()

Remove files in the download directory that have filesize equal to 0

setDirectoryIn(day)

Enter into the file directory of a specified day

Parameters:day (str) – a string representing a day in format YYYY.MM.DD

setDirectoryOver()

Move up within the file directory

pymodis.downmodis.getNewerVersion(oldFile, newFile)

Check two files to determine which is newer

Parameters:

• oldFile (str) – one of the two similar files
• newFile (str) – one of the two similar files

Returns:

the name of newer file

class pymodis.downmodis.modisHtmlParser(fh)

Bases: HTMLParser.HTMLParser

A class to parse HTML

Parameters:fh – content of http request

get_all()

Return everything

get_dates()

Return a list of directories with date

get_tiles(prod, tiles, jpeg=False)

Return a list of files to download

Parameters:

• prod (str) – the code of MODIS product that we are going to analyze
• tiles (list) – the list of tiles to consider
• jpeg (bool) – True to also check for jpeg data

handle_starttag(tag, attrs)

pymodis.downmodis.str2date(datestring)

Convert to datetime.date object from a string

:param str datestring string with format (YYYY-MM-DD)

Returns:a datetime.date object representing datestring

pymodis.downmodis.urljoin(*args)

Joins given arguments into a url. Trailing but not leading slashes are stripped for each argument. http://stackoverflow.com/a/11326230

Returns:a string