refactored to use google places search url instead of local services listing which is now not available#7
Merged
christivn merged 2 commits intochristivn:mainfrom Apr 15, 2026
Conversation
- Created a new `mapScraper` package with an `__init__.py` for better structure. - Implemented `places_crawler.py` which now uses different url for seatching of Google Maps Places. - Updated `mapScraperX.py` to support new scraping modes: fast, standard, and complete. - Updated `requirements.txt` to include `phonenumbers` for phone validation.
Owner
|
Hi @devsanthoshmk , thank you so much for your contribution to the project. Great work! I don't have much time to write code right now, but I hope to see more pull requests from you with new features or improvements for the project 💪 Thx <3 |
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I implemented the scraper using the discussed methods in #5. I wrote the core scrapping logic and methods on my own and used claude code implement that part in mapScraper/placesCrawlerV2.py and i read every line of the code. everything is working to my eyes and i can change if you want anything to. Sorry for the commit history i was using this in some other places so i just pasted them here instead of preserving the commit history.
Here is how i implemented it:
I get the search results from here
https://www.google.com/search?q={query}&start={pagination}&udm=1&hl={lang}&gl={country}https://www.google.com/async/lcl_akp?q={query}&async=ludocids:{cid},_fmt:progwith the cid from the previous result by which we can get phone numbers, complete address and website link of the places(if phone number is not there then website is also not present).fast,strandard,complete(refer readme for more info on this) which will handle the missing phone number problem.