As you can see in this trace
each notifier instance is doing a bunch of duplicated work:
- It is re-requesting a github token over and over again, adding to our rate limiting problem
- It is querying (presumably the same)
Commit over and over again
- The
SELECT/INSERT of the commit_notifications could most likely be optimized by batching across all the various status notifications to be emitted
POST-ing the notifications to github should ideally happen in a batch (if the API allows it), instead of doing it one-by-one
Another optimization opportunity:
We have a redis GET immediately followed by SETEX is likely some form of cache. However it has the following problems:
GET with an immediate SETEX indicates a cache miss, so the cache might be ineffective- There is no visible gap / span in between the
GET and SETEX, so whatever operation is being cached is likely very fast and does not need caching in the first place.
As you can see in this trace
each notifier instance is doing a bunch of duplicated work:
Commitover and over againSELECT/INSERTof thecommit_notificationscould most likely be optimized by batching across all the various status notifications to be emittedPOST-ing the notifications to github should ideally happen in a batch (if the API allows it), instead of doing it one-by-oneAnother optimization opportunity:
We have a redis
GETimmediately followed bySETEXis likely some form of cache. However it has the following problems:GETwith an immediateSETEXindicates a cache miss, so the cache might be ineffectiveGETandSETEX, so whatever operation is being cached is likely very fast and does not need caching in the first place.