Search local data directory

[digizeph]

Specify a local directory containing MRT files, and it should be able to search across all of them with the same filtering parameters. We can potentially expand the monocle parse subcommand to achieve this feature.

[jwbensley]

+1 for this

[digizeph]

lol, I can't believe its been 3 years since I created this issue. I guess there was not many real usages for this feature until now.

For an initial design, I'd propose to have a new parameter to specify a MRT cache directory. The files needed for parsing will be downloaded to the dir with files named by roughly the same structure as the original files: cache_dir/PROJECT/COLLECTOR/YEAR/MONTH/file.

With cache directory specified, monocle should first attempt to identify files needed from the local dir before attempt to retrieve from remote. The parsing of the remote file will also be changed from streaming to download-then-parse process.