Skip to content

Conversation

@dev-warrior777
Copy link
Contributor

Completes #3383

  • Removed tools configuration

Some more cleanup to do before review.

* Removed tools configuration
Copy link
Member

@JoeGruffins JoeGruffins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry reviewing before you are ready.

So, what's the flow? I guess I would expect:

  1. Try to get the hashes file at https://www.getmonero.org/downloads/hashes.txt and verify that with some public key somewhere. I guess we check every start up? Anyway when they first download for now is fine.

  2. Parse the list and pick out the version and hash of the os and arch we should have.

  3. Check if we have that version. If yes done. If no continue. Also continue if we have nothing downloaded yet.

  4. Further check we can use the version in the list. We would accept any version v0.n.n.x. x, or patch can be higher than what we have hard coded. All other numbers must be the same. If we already have a valid version and the one in the list is not valid, stop and do not update.

  5. If we have nothing and the version from the list is not valid, we must have a hard coded version and hash to download an earlier version and continue. We would also jump here if we had a problem getting or reading the list and have nothing.

  6. Download the file and check hash.

  7. Decompress to where we will keep it.

  8. Use.

@buck54321
Copy link
Member

This seems like a bad idea to me. If people want a full node wallet, they can run a node and handle the rpc api configuration through the ui. I thought we were implementing a monero light wallet first.

@JoeGruffins
Copy link
Member

@buck54321 it's like running dcrwallet but dcrd is remote. One can set up a full node to connect to as you say.

I think it is a light wallet. please see https://docs.getmonero.org/interacting/monero-wallet-rpc-reference/#json-rpc-interface

Wallet uses your private keys to understand your total balance, transactions history, and to facilitate creating transactions.

However, wallet does not store the blockchain and does not directly participate in the p2p network.

We don't have any golang libraries to use this technology is why we are downloading the binaries for now.

* Removed tools from configuration settings dialog
* Tightened up the zip-matching logic to map golang machine to zip file name
* Updated tests: zip downloading, wallet creation, moneroVersionV0
@dev-warrior777
Copy link
Contributor Author

So, what's the flow? I guess I would expect:

Just follow Download.Run func. It is essentially laid out as the steps .. rejection of invalid version is handled by moneroVersionV0 - reject if < v0.18.4.3 or does not start with 'v0' since it never has in monero recent history.

Since this all happens only in create and we always want the latest version from hashes.txt there is no initial check for "have we got it already" since we almost always do not.

However there are exported functions to do those kind of checks .. mostly for later.

@dev-warrior777
Copy link
Contributor Author

dev-warrior777 commented Dec 12, 2025

Sorry reviewing before you are ready.

No that's fine. It gives me the chance to explain where I am going.

Run download_test.go

This is the relevant part of hashes.txt:

## CLI
7c2ad18ca3a1ad5bc603630ca935a753537a38a803e98d645edd6a3b94a5f036  monero-android-armv7-v0.18.4.4.tar.bz2
eb81b71f029884ab5fec76597be583982c95fd7dc3fc5f5083a422669cee311e  monero-android-armv8-v0.18.4.4.tar.bz2
bc539178df23d1ae8b69569d9c328b5438ae585c0aacbebe12d8e7d387a745b0  monero-freebsd-x64-v0.18.4.4.tar.bz2
2040dc22748ef39ed8a755324d2515261b65315c67b91f449fa1617c5978910b  monero-linux-armv7-v0.18.4.4.tar.bz2
b9daede195a24bdd05bba68cb5cb21e42c2e18b82d4d134850408078a44231c5  monero-linux-armv8-v0.18.4.4.tar.bz2
c939ea6e8002798f24a56ac03cbfc4ff586f70d7d9c3321b7794b3bcd1fa4c45  monero-linux-riscv64-v0.18.4.4.tar.bz2
7fe45ee9aade429ccdcfcad93b905ba45da5d3b46d2dc8c6d5afc48bd9e7f108  monero-linux-x64-v0.18.4.4.tar.bz2
8c174b756e104534f3d3a69fe68af66d6dc4d66afa97dfe31735f8d069d20570  monero-linux-x86-v0.18.4.4.tar.bz2
645e9bbae0275f555b2d72a9aa30d5f382df787ca9528d531521750ce2da9768  monero-mac-armv8-v0.18.4.4.tar.bz2
af3d98f09da94632db3e2f53c62cc612e70bf94aa5942d2a5200b4393cd9c842  monero-mac-x64-v0.18.4.4.tar.bz2
7eb3b87a105b3711361dd2b3e492ad14219d21ed8fd3dd726573a6cbd96e83a6  monero-win-x64-v0.18.4.4.zip
a148a2bd2b14183fb36e2cf917fce6f33fb687564db2ed53193b8432097ab398  monero-win-x86-v0.18.4.4.zip
84570eee26238d8f686605b5e31d59569488a3406f32e7045852de91f35508a2  monero-source-v0.18.4.4.tar.bz2
#

@JoeGruffins
Copy link
Member

Just follow Download.Run func.

Can you humor me. Is it very different than what I would expect and why?

Since this all happens only in create and we always want the latest version from hashes.txt there is no initial check for "have we got it already"

It would be done from different networks and different build dirs. It's one dir forbthe user currently. We also don't want latest of the Major version is higher.

@JoeGruffins
Copy link
Member

Itvlooks like the main difference is that you aren't checking if we already have it. If I made a simnet wallet then a mainnet wallet I would already have it. If we add a wallet delete like btc we would already have it. if restoring feom seed may already have it.

@dev-warrior777
Copy link
Contributor Author

dev-warrior777 commented Dec 12, 2025

It looks like the main difference is that you aren't checking if we already have it. If I made a simnet wallet then a mainnet wallet I would already have it. If we add a wallet delete like btc we would already have it. if restoring feom seed may already have it.

Even if we have 0.18.4.3 we want to update to 0.18.4.4 (and for now) the old one would not be overwritten)

I see no circumstance where we would have it before wallet creation
Added a test latestToolsNeeded before starting a download

@dev-warrior777
Copy link
Contributor Author

Can you humor me.

I did not mean like that ...

The Download.Run function drives the sequencing and is a pure list of steps for the download/extract process. Scanning that was my way of pointing you to an easy way to grok the download process.

* Added a tools status check:
  - do we already have latest good tools version locally
  - is a higher version available remotely & thus 'needed'

* moved base dir to [user dir].dexc/share/monero-tools
* Tools path now dynamic based on cfg.DataDir
* Updated tests
* Updated potential panic
* Versioning: (minimum) v0.18.4.3 <= valid < v0.19.x.x
* Suggested making a json policy object to control later monero
  version updates with a json file.
@dev-warrior777
Copy link
Contributor Author

Yeah, not quite right .. I really need --appdata but only DataDir available at Driver.Create time. Working ..

* Re-wrote getToolsBasePath - still based on DataDir.
* Updated all download tests and added TestToolsBasePath.
@dev-warrior777
Copy link
Contributor Author

Got a couple minor edge cases for architectures/OS's I want to look at but I think ready to review

@dev-warrior777 dev-warrior777 marked this pull request as ready for review December 16, 2025 16:36
Copy link
Member

@JoeGruffins JoeGruffins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

@dev-warrior777
Copy link
Contributor Author

dev-warrior777 commented Dec 25, 2025

The MAV stuff can be easily pulled out if it is not what you meant.

Another way of hard coding an acceptable version would be to package one version of the monero zip set in dex. We need 7 os/arch files to cover the intersection of os/arch's we support in dex so 7x80MB for the full zipped set. But could also make a versioned dir of the extracted tools which would be 7x56MB which is actually quite large due to good compression I guess.

Copy link
Member

@JoeGruffins JoeGruffins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very involved, looks like you thought of every angle here. Just a few more nits.

// monero-android-armv7-v0.18.4.3.tar.bz2, 4e1481835824b9233f204553d4a19645274824f3f6185d8a4b50198470752f54
// monero-freebsd-x64-v0.18.4.3.tar.bz2, ff7b9c5cf2cb3d602c3dff1902ac0bc3394768cefc260b6003a9ad4bcfb7c6a4

var jsonVersions = `
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks tedious to create. Can you also create a small tool to regenerate these and spit them to stdout for when we would update them?

Copy link
Contributor Author

@dev-warrior777 dev-warrior777 Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure ..

Do you know a good place to put the tool if we do not want to maintain an extra go.mod?

Or such a tool could be in a test file?

I see the easier process you suggest as:

  • Go to github monero releases page(s)
  • Copy and paste one or several version lists of hashed zip definitions
    (as shown at top of ma_version.go but could be several versions in the set if needed as the json struct is an array of objects which contain an array of "zips")
  • Insert "## CLI" on a line above the first or only set of hashed zips
  • Insert "#" in between version lists
  • Put all the lines into a file called e.g. input
  • Let the tool read input make structs and marshall them .. maybe add backticks
  • Feed the json to stdout

What do you think ..?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see JoeGruffins@745c3f0

The tool is small. That code breaks some other things but I hope you can see how it works. Outputs:

hashes output
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

# This GPG-signed message exists to confirm the SHA256 sums of Monero binaries.
#
# Please verify the signature against the key for binaryFate in the
# source code repository (/utils/gpg_keys).
#
#
## CLI
7c2ad18ca3a1ad5bc603630ca935a753537a38a803e98d645edd6a3b94a5f036  monero-android-armv7-v0.18.4.4.tar.bz2
eb81b71f029884ab5fec76597be583982c95fd7dc3fc5f5083a422669cee311e  monero-android-armv8-v0.18.4.4.tar.bz2
bc539178df23d1ae8b69569d9c328b5438ae585c0aacbebe12d8e7d387a745b0  monero-freebsd-x64-v0.18.4.4.tar.bz2
2040dc22748ef39ed8a755324d2515261b65315c67b91f449fa1617c5978910b  monero-linux-armv7-v0.18.4.4.tar.bz2
b9daede195a24bdd05bba68cb5cb21e42c2e18b82d4d134850408078a44231c5  monero-linux-armv8-v0.18.4.4.tar.bz2
c939ea6e8002798f24a56ac03cbfc4ff586f70d7d9c3321b7794b3bcd1fa4c45  monero-linux-riscv64-v0.18.4.4.tar.bz2
7fe45ee9aade429ccdcfcad93b905ba45da5d3b46d2dc8c6d5afc48bd9e7f108  monero-linux-x64-v0.18.4.4.tar.bz2
8c174b756e104534f3d3a69fe68af66d6dc4d66afa97dfe31735f8d069d20570  monero-linux-x86-v0.18.4.4.tar.bz2
645e9bbae0275f555b2d72a9aa30d5f382df787ca9528d531521750ce2da9768  monero-mac-armv8-v0.18.4.4.tar.bz2
af3d98f09da94632db3e2f53c62cc612e70bf94aa5942d2a5200b4393cd9c842  monero-mac-x64-v0.18.4.4.tar.bz2
7eb3b87a105b3711361dd2b3e492ad14219d21ed8fd3dd726573a6cbd96e83a6  monero-win-x64-v0.18.4.4.zip
a148a2bd2b14183fb36e2cf917fce6f33fb687564db2ed53193b8432097ab398  monero-win-x86-v0.18.4.4.zip
84570eee26238d8f686605b5e31d59569488a3406f32e7045852de91f35508a2  monero-source-v0.18.4.4.tar.bz2
#
## GUI
4c81c8e97bd542daa453776d888557db1ceb2a718d43f6135ad68b12c8119948  monero-gui-install-win-x64-v0.18.4.4.exe
e45cb3fa9d972d67628cfed6463fb7604ae1414a11ba449f5e2f901c769ac788  monero-gui-linux-x64-v0.18.4.4.tar.bz2
a6f071719c401df339dba2d43ec6fffe103fda3e1df46f354b2496f34bb61cc4  monero-gui-mac-armv8-v0.18.4.4.dmg
811df70811a25f31289f24ebc0edc8f7648670384698d4c768bac5c2acbf2026  monero-gui-mac-x64-v0.18.4.4.dmg
b96faa56aa77cabed1f31f3fc9496e756a8da8c1124da2b9cb0b3730a8b6fbd9  monero-gui-win-x64-v0.18.4.4.zip
a7f6b91bc9efaa83173a397614626bf7612123e0017a48f66137ac397f7d19f8  monero-gui-source-v0.18.4.4.tar.bz2
#
#
# ~binaryFate
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEgaxZH+nEtlxYBq/D8K9NRioL35IFAmkbGLgACgkQ8K9NRioL
35LWYRAAnPeUu7TADV9Nly2gBlwu7bMK6l7pcUzs3hHhCMpg/Zb7wF8lx4D/r/hT
3wf3gNVK6tYl5GMPpF7GSKvK35SSzNN+8khRd7vhRByG75LGLnrNlcBsQU2wOzUv
Rmm2R8L8GP0B/+zXO92uJDMZ7Q7x72O+3fVX05217HBwz2kvzE1NpXe+EJPnUukA
Tr5CRnxKhxPbilvIhoEHdwkScMZqHMfsbdrefrB3KpO3xEaUz+gO9wESp7nzr4vp
Du6gJYBPK25Z2heZHCRsGN4WQP4QQv4MC0IFczc9fkVDBjywsJeNRRUbGtxR/BNt
vNJGI/kS+7KV140j6GkqAh/leZcaVJ5LRyCaHAwEQNA2T5okhrM0WZpoOAsZMi5K
bW4lNOXfWSw6/tokEPeuoi49yw0f9z0C8a4VLNOZGWKqmHcsA8WE6oVfmvVk6xWu
BqTU1Z9LJqL17GWRAReSX1ZuNA0Q0Pb/klUwP4X2afJcCVZ2YeBNr4jr21u3dYXY
QiLj0Gv7gg7a/GiMpVglNn5GzCu6mT0D94sbMNK+U5Tbve7aOtijJZ8JR62eO/mR
h+oNEys/xEcP9PQ5p74cNL71hNSfWSOcNi+GLSgXC75vsOGr7i96uaamilsHnsYB
p8PZMHzOf1pi6i/L5oOEuRgaujd9IjyCbxoYh3bbxxjBOhNEMqU=
=CVLA
-----END PGP SIGNATURE-----

[
  {
    "hash": "7c2ad18ca3a1ad5bc603630ca935a753537a38a803e98d645edd6a3b94a5f036",
    "zip": "monero-android-armv7-v0.18.4.4.tar.bz2",
    "dir": "monero-android-armv7-v0.18.4.4",
    "ext": "",
    "os": "",
    "arch": ""
  },
  {
    "hash": "eb81b71f029884ab5fec76597be583982c95fd7dc3fc5f5083a422669cee311e",
    "zip": "monero-android-armv8-v0.18.4.4.tar.bz2",
    "dir": "monero-android-armv8-v0.18.4.4",
    "ext": "",
    "os": "",
    "arch": ""
  },
  {
    "hash": "bc539178df23d1ae8b69569d9c328b5438ae585c0aacbebe12d8e7d387a745b0",
    "zip": "monero-freebsd-x64-v0.18.4.4.tar.bz2",
    "dir": "monero-freebsd-x64-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "freebsd",
    "arch": "x64"
  },
  {
    "hash": "2040dc22748ef39ed8a755324d2515261b65315c67b91f449fa1617c5978910b",
    "zip": "monero-linux-armv7-v0.18.4.4.tar.bz2",
    "dir": "monero-linux-armv7-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "linux",
    "arch": "armv7"
  },
  {
    "hash": "b9daede195a24bdd05bba68cb5cb21e42c2e18b82d4d134850408078a44231c5",
    "zip": "monero-linux-armv8-v0.18.4.4.tar.bz2",
    "dir": "monero-linux-armv8-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "linux",
    "arch": "armv8"
  },
  {
    "hash": "c939ea6e8002798f24a56ac03cbfc4ff586f70d7d9c3321b7794b3bcd1fa4c45",
    "zip": "monero-linux-riscv64-v0.18.4.4.tar.bz2",
    "dir": "monero-linux-riscv64-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "linux",
    "arch": "riscv64"
  },
  {
    "hash": "7fe45ee9aade429ccdcfcad93b905ba45da5d3b46d2dc8c6d5afc48bd9e7f108",
    "zip": "monero-linux-x64-v0.18.4.4.tar.bz2",
    "dir": "monero-linux-x64-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "linux",
    "arch": "x64"
  },
  {
    "hash": "8c174b756e104534f3d3a69fe68af66d6dc4d66afa97dfe31735f8d069d20570",
    "zip": "monero-linux-x86-v0.18.4.4.tar.bz2",
    "dir": "monero-linux-x86-v0.18.4.4",
    "ext": "",
    "os": "",
    "arch": ""
  },
  {
    "hash": "645e9bbae0275f555b2d72a9aa30d5f382df787ca9528d531521750ce2da9768",
    "zip": "monero-mac-armv8-v0.18.4.4.tar.bz2",
    "dir": "monero-mac-armv8-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "mac",
    "arch": "armv8"
  },
  {
    "hash": "af3d98f09da94632db3e2f53c62cc612e70bf94aa5942d2a5200b4393cd9c842",
    "zip": "monero-mac-x64-v0.18.4.4.tar.bz2",
    "dir": "monero-mac-x64-v0.18.4.4",
    "ext": "tar.bz2",
    "os": "mac",
    "arch": "x64"
  },
  {
    "hash": "7eb3b87a105b3711361dd2b3e492ad14219d21ed8fd3dd726573a6cbd96e83a6",
    "zip": "monero-win-x64-v0.18.4.4.zip",
    "dir": "monero-win-x64-v0.18.4.4",
    "ext": "zip",
    "os": "win",
    "arch": "x64"
  },
  {
    "hash": "a148a2bd2b14183fb36e2cf917fce6f33fb687564db2ed53193b8432097ab398",
    "zip": "monero-win-x86-v0.18.4.4.zip",
    "dir": "monero-win-x86-v0.18.4.4",
    "ext": "",
    "os": "",
    "arch": ""
  },
  {
    "hash": "84570eee26238d8f686605b5e31d59569488a3406f32e7045852de91f35508a2",
    "zip": "monero-source-v0.18.4.4.tar.bz2",
    "dir": "monero-source-v0.18.4.4",
    "ext": "",
    "os": "",
    "arch": ""
  }
]

Copy link
Contributor Author

@dev-warrior777 dev-warrior777 Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... needs to do many not just one { [ { [hzips] }, { [hzips ] }, ... ] }

I need to re-use some of the already written code .. so where to put the main()

Copy link
Contributor Author

@dev-warrior777 dev-warrior777 Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we put in new cmd then new go.mod dependency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now, you want to grab an older version to parse? I think it's fine to just use getmonero.org and if there is some problem with doing that, we can fall back to manual.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can also do this in another pr if you don't want to.

Copy link
Contributor Author

@dev-warrior777 dev-warrior777 Jan 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have 80% done it - with a conflict that needs to be resolved and marshalling yet to do - can already grab info on the last 9 releases from the first page .. no need go back further than v0.18.3.x.

See what you think on this possibility ..

Note: All the MAV and to a lesser extent the tool was written to be relatively easy to take out and I stashed this commit with 80% tool in a full dcrdex file tree .. so minds can change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I need a go.mod in xmr/cmd? no rite?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you don't need a go.mod file. For some reason we only use one main one. I mean... you can make one or not I guess whatever is fine.

dev-warrior777 added 5 commits December 30, 2025 00:05
* Worried a bit about what the compiler does with the defers in the case statement; then did the research.
* Go to monero github releases page
* Download and parse HTML
* Make JSON for last 9 CLI releases
* JSON tool now useful for future downloads
* Hopefully weird go.mod issue fixed (no local issue)
* Added mkJson tool README.md
@dev-warrior777
Copy link
Contributor Author

dev-warrior777 commented Jan 2, 2026

Then I tried adding a go.mod in xmr/cmd/mkJson:

errors
~/dex/dcrdex/client/asset/xmr/cmd/mkjson (xmr-tools-dl)$ cat go.mod

module decred.org/dcrdex/client/asset/xmr/cmd/mkjson

go 1.25.4
~/dex/dcrdex/client/asset/xmr/cmd/mkjson (xmr-tools-dl)$ go mod tidy

go: finding module for package decred.org/dcrdex/client/asset/xmr/toolsdl
go: finding module for package github.com/decred/slog
go: finding module for package decred.org/dcrdex/dex/dexnet
go: finding module for package github.com/PuerkitoBio/goquery
go: finding module for package decred.org/dcrdex/dex
go: downloading decred.org/dcrdex v1.0.5
go: found decred.org/dcrdex/dex in decred.org/dcrdex v1.0.5
go: found decred.org/dcrdex/dex/dexnet in decred.org/dcrdex v1.0.5
go: found github.com/PuerkitoBio/goquery in github.com/PuerkitoBio/goquery v1.11.0
go: found github.com/decred/slog in github.com/decred/slog v1.2.0
go: finding module for package decred.org/dcrdex/client/asset/xmr/toolsdl
go: decred.org/dcrdex/client/asset/xmr/cmd/mkjson imports
	decred.org/dcrdex/client/asset/xmr/toolsdl: module decred.org/dcrdex@latest found (v1.0.5), but does not contain package decred.org/dcrdex/client/asset/xmr/toolsdl

So that will not work. I guess because not merged yet?

I will try to find another html->text tool ..

Edit: Found anaskhan96/soup which can manipulate HTML DOM

* Use anaskhan96/soup to strip tags in HTML
* No separate go.mod
* General tidy up
* Test monero exchange wallet still working
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants