drivers/mtd: introduce nvblk

[Laczen]

Summary

NVBLK provides a block device that operates on top of a non volatile memory (as a mtd device) that enables wear levelling for non volatile memory. It's operation is similar to the dhara wear levelling library, but nvblk is meant to be used on smaller (nor, mram, rram) memory.

I am also the author and maintainer of the nvblk library.

A block device can be created during startup by using:

nvblk_initialize("/dev/mtdblock0", mtd,
                 CONFIG_MTD_NVBLK_DEFAULT_LBS,
                 CONFIG_MTD_NVBLK_DEFAULT_IOBS,
                 CONFIG_MTD_NVBLK_DEFAULT_SPEB);

and a fat filesystem on top of this as:

nsh> mkfatfs /dev/mtdblock0
nsh> mount -t vfat /dev/mtdblock0 /mnt

this fat filesystem can then be used:

nsh> echo "test" > /mnt/test.txt
nsh> cat test.txt
test
nsh>

Impact

Impact on users: none, new type of block device is provided for future use
Impact on build proces: none, new type of block device is provided for future use
Impact on hardware: none,
Impact on documentation: none (a sample config will be added) sample defconfig added,
Impact on security: none,
Impact on compatibility: none

Testing

The correct operation of this block device was verified on esp32-devkitc.
The block device was tested using NuttX vfat library and littlefs library.

[Laczen]

@acassis @xiaoxiang781216 I am having some problems in creating a Make.defs that does a proper distclean. It seems that the folder nvblk is kept under mtd after distclean. I have seen that the same problem appears for littlefs. What would be the correct way to do a proper distclean ?

[xiaoxiang781216]

@acassis @xiaoxiang781216 I am having some problems in creating a Make.defs that does a proper distclean. It seems that the folder nvblk is kept under mtd after distclean. I have seen that the same problem appears for littlefs. What would be the correct way to do a proper distclean ?

the script in Makefile looks good to me:(.

[Laczen]

@xiaoxiang781216 first I would like to thank you for reviewing this PR so fast.

Regarding the mtd_proxy approach I still have my doubts: when a flash wear levelling library like dhara or nvblk is inserted the direct link between the mtd and block device is broken. And I don't think it is possible to use direct mtd writing/reading and block writing/reading at the same time. Suppose you would like to dd into the mtd device resp. the block device, they have a complete different meaning: dd into the mtd device would be a change on the hardware directly, dd into the block device would be writing blocks. How to differentiate between these two ?

I think the same problem might exist for nand flashes (but I haven't checked the ftl thoroughly).

I remember that you made a remark that ftl_initialize should not be called from a board startup, but I didn't quite understand this. Could you explain this a little more?

[xiaoxiang781216]

@xiaoxiang781216 first I would like to thank you for reviewing this PR so fast.

Regarding the mtd_proxy approach I still have my doubts: when a flash wear levelling library like dhara or nvblk is inserted the direct link between the mtd and block device is broken. And I don't think it is possible to use direct mtd writing/reading and block writing/reading at the same time.

Yes, the direct mode isn't suitable for the advanced wear leveling algo like dhara/nvblk since both save the meta data into flash. The direct mode will destroy this critical information.

Suppose you would like to dd into the mtd device resp. the block device, they have a complete different meaning: dd into the mtd device would be a change on the hardware directly, dd into the block device would be writing blocks. How to differentiate between these two ?

I think the same problem might exist for nand flashes (but I haven't checked the ftl thoroughly).

But since ftl doesn't save any meta data into flash, so the direct mode(skip buffer and erase) works well with it in some special user case.

I remember that you made a remark that ftl_initialize should not be called from a board startup, but I didn't quite understand this. Could you explain this a little more?

That's because fs layer will add FTL wrapper on mtd device automatically when the user open /dev/mtdblockx, which equal to you call ftl_initialize in board file manually.

[Laczen]

I think the same problem might exist for nand flashes (but I haven't checked the ftl thoroughly).

But since ftl doesn't save any meta data into flash, so the direct mode(skip buffer and erase) works well with it in some special user case.

After going through the ftl.c code the correct operation is indeed limited to some special user case. The mapping from logical blocks to physical good blocks breaks the relation between the mtd addresses (that should include the bad blocks) and the block based addresses (that do not include the bad blocks). This makes the addressing ambiguous.

[Laczen]

@acassis, @lupyuen you helped me in the past for a CI failure. I need some help again. I tried searching for "err" in the CI output but could not find anything, so I have no clue about what is going on. Thanks in advance.

[xiaoxiang781216]

@acassis, @lupyuen you helped me in the past for a CI failure. I need some help again. I tried searching for "err" in the CI output but could not find anything, so I have no clue about what is going on. Thanks in advance.

you need remove all files under drivers/mtd/nvblk/ which isn't added to git when user issue make distclean.

[Laczen]

@acassis, @lupyuen you helped me in the past for a CI failure. I need some help again. I tried searching for "err" in the CI output but could not find anything, so I have no clue about what is going on. Thanks in advance.

you need remove all files under drivers/mtd/nvblk/ which isn't added to git when user issue make distclean.

@xiaoxiang781216 thanks for the reply. I am unsure if I understand you correct. Do you mean that it is required that make distclean removes all files under /mtd/nvblk ?

[xiaoxiang781216]

yes, the file download/unzip and not add to git.

[Laczen]

@xiaoxiang781216 I think I have found the problem in the Make.defs: when make distclean is called the config CONFIG_MTD_NVBLK is no longer set and the specific distclean added for this config is not called this was not the problem. I will try to rework the Make.defs.