Conversation
added 6 commits
October 12, 2025 17:48
Port the hash() function from JavaScript to Go. This function generates an MD5 hash of an article's GUID. - Created go/feeds package structure - Implemented Hash() function using crypto/md5 - Added comprehensive tests using YAML test data - All 3 hash function tests passing ✓
Port the score() function from JavaScript to Go. This function generates a timestamp score for articles based on their publication date. - Implemented Score() function using time.Parse and time.UnixMilli - Handles pubDate, pubdate, and date fields in priority order - Falls back to current time for invalid dates - All 4 score function tests passing ✓
Port all remaining feed utility functions from JavaScript to Go: - BuildRequestHeaders: Build HTTP headers for conditional GET requests - BuildRedisKeys: Generate Redis key names for feeds - BuildArticleKey: Build article keys for Redis sorted sets - ProcessArticle: Add computed fields (hash, score, feedurl) to articles - ShouldStoreArticle: Determine if article needs S3 storage - IsValidArticle: Validate article has required fields - ExtractArticleIds: Extract article IDs from Redis keys - GenerateArticleBody: Generate JSON body for S3 storage All 36 tests passing ✓ (3 hash + 4 score + 29 utils)
Added separate test-go job to run Go tests in CI: - Tests run on Go 1.22 and 1.23 - Runs all tests in go/feeds/ package - Runs in parallel with Node.js tests
Renamed go/feeds to go/feedfetcher for clarity and consistency. Updated all package declarations and CI workflow accordingly. All 36 tests still passing ✓
Extracted hardcoded test data from both JS and Go tests into shared YAML files in testdata/ directory. This ensures both test suites use exactly the same test data. Changes: - Added new test data to testdata/test-cases.yaml: - redis_key_prefix_tests - build_article_key_tests - process_article_tests - extract_article_ids_tests - generate_article_body_tests - Updated src/lib/feedUtils.test.js to use YAML data for all tests - Updated feedfetcher/utils_test.go to use YAML data for all tests All tests passing: - JavaScript: 29 tests ✓ (7 articleUtils + 22 feedUtils) - Go: 38 tests ✓ (7 articles + 31 utils)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR ports all JavaScript feed utility functions to Go in preparation for migrating the feed fetcher service.
Changes
New Go Package:
feedfetcherAll utility functions have been ported with comprehensive test coverage using the same YAML test data:
Article Utilities (articles.go):
Hash(): Generate MD5 hash of article GUIDScore(): Generate timestamp score from publication dateFeed Utilities (utils.go):
BuildRequestHeaders(): Build HTTP headers for conditional GET requestsBuildRedisKeys(): Generate Redis key names for feedsBuildArticleKey(): Build article keys for Redis sorted setsProcessArticle(): Add computed fields (hash, score, feedurl)ShouldStoreArticle(): Determine if article needs S3 storageIsValidArticle(): Validate article has required fieldsExtractArticleIds(): Extract article IDs from Redis keysGenerateArticleBody(): Generate JSON body for S3 storageTest Coverage
testdata/test-cases.yamlCommit Structure
Each function was committed separately with its tests:
Testing
All tests pass locally and use the same test data as the JavaScript version:
```bash
cd feedfetcher
go test -v
```
🤖 Generated with Claude Code