concat dataset

[yueyinqiu]

• add support for torch.utils.data.ConcatDataset
  • a new interface IDataset<out T> is added
    • now Dataset<T> implements IDataset<T>
  • more overloads of DataLoader() has been added, to accept IDataset datasets
    • those overloads directly return a DataLoader<T, S>, rather than its subclasses
  • datasets supported by DataLoader<T, S> have been relaxed to IDataset<T>
• parameter of collate functions in DataLoader<T, S> has been relaxed to IReadOnlyList

[yueyinqiu]

related to #1348 (comment) #1354 #1358

[yueyinqiu]

As mentioned here #1357 (comment), the current version has no API changes (except the type of dataset and collate_fn in DataLoader<T, S>).

However we don't actually need Dataset<T>, Dataset and IterableDataset. Do we have any plan to remove them later? They may also occupy the position of some other classes in PyTorch, like #1353.

(But I find it hard to use [Obsolete] on them since some methods are using them as the return type.)

[yueyinqiu]

Here we are using Int64. However actually most .net containers does not support Int64 indices. And actually it is impractical to have so much data. Should we switch to Int32 instead?

[K1T00]

What is the status on this one? I waiting for this to be merged.

[yueyinqiu]

Perhaps you could implement one yourself... This PR is one year old, and I'm not even sure if it conflicts with existing code...