Skip to content

Commit 7fbb0eb

Browse files
sshawsshaw
committed
Remove use a ShopAPIRetry and add exponential backoff support
1 parent 5aae43f commit 7fbb0eb

7 files changed

Lines changed: 666 additions & 110 deletions

File tree

‎.env.template‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
SHOPIFY_DOMAIN=
1010
SHOPIFY_TOKEN=
1111

12-
# Will result in the creation of a private metadafield
12+
# Will result in the creation of a metadafield on the customer. write_customers permission required
1313
SHOPIFY_CUSTOMER_ID=
1414

1515
# Must have more than 1 variant values

‎README.md‎

Lines changed: 72 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ShopifyAPI::GraphQL::Tiny
22

3-
Lightweight, no-nonsense, Shopify GraphQL Admin API client with built-in pagination and retry.
3+
Lightweight, no-nonsense, Shopify GraphQL Admin API client with built-in pagination and retry
44

55
[![CI](https://github.com/ScreenStaring/shopify_api-graphql-tiny/actions/workflows/ci.yml/badge.svg)](https://github.com/ScreenStaring/shopify_api-graphql-tiny/actions)
66

@@ -74,6 +74,69 @@ GQL
7474
p result.dig("data", "customerUpdate", "userErrors")
7575
```
7676

77+
### Automatically Retrying Failed Requests
78+
79+
There are 2 types of retries: 1) request is rate-limited by Shopify 2) request fails due to an exception or non-200 HTTP response.
80+
81+
When a request is rate-limited by Shopify retry occurs according to [Shopify's `throttleStatus`](https://shopify.dev/docs/api/admin-graphql/unstable#rate-limits)
82+
83+
When a request fails due to an exception or non-200 HTTP status a retry will be attempted after an exponential backoff waiting period.
84+
This is controlled by `ShopifyAPI::GraphQL::Tiny::DEFAULT_BACKOFF_OPTIONS`. It contains:
85+
86+
* `:base_delay` - `0.5`
87+
* `:jitter` - `true`
88+
* `:max_attempts` - `10`
89+
* `:max_delay` - `60`
90+
* `:multiplier` - `2.0`
91+
92+
`:max_attempts` dictates how many retry attempts will be made **for all** types of retries.
93+
94+
These can be overridden globally (by assigning to the constant) or per instance:
95+
96+
```rb
97+
gql = ShopifyAPI::GraphQL::Tiny.new(shop, token, :max_attempts => 20, :max_delay => 90)
98+
```
99+
100+
`ShopifyAPI::GraphQL::Tiny::DEFAULT_RETRY_ERRORS` determines what is retried. It contains and HTTP statuses codes, Shopify GraphQL errors codes, and exceptions.
101+
By default it contains:
102+
103+
* `"5XX"` - Any HTTP 5XX status
104+
* `"INTERNAL_SERVER_ERROR"` - Shopify GraphQL error code
105+
* `"TIMEOUT"` - Shopify GraphQL error code
106+
* `EOFError`
107+
* `Errno::ECONNABORTED`
108+
* `Errno::ECONNREFUSED`
109+
* `Errno::ECONNRESET`
110+
* `Errno::EHOSTUNREACH`
111+
* `Errno::EINVAL`
112+
* `Errno::ENETUNREACH`
113+
* `Errno::ENOPROTOOPT`
114+
* `Errno::ENOTSOCK`
115+
* `Errno::EPIPE`
116+
* `Errno::ETIMEDOUT`
117+
* `Net::HTTPBadResponse`
118+
* `Net::HTTPHeaderSyntaxError`
119+
* `Net::ProtocolError`
120+
* `Net::ReadTimeout`
121+
* `OpenSSL::SSL::SSLError`
122+
* `SocketError`
123+
* `Timeout::Error`
124+
125+
These can be overridden globally (by assigning to the constant) or per instance:
126+
127+
```rb
128+
# Only retry on 2 errors
129+
gql = ShopifyAPI::GraphQL::Tiny.new(shop, token, :retry => [SystemCallError, "500"])
130+
```
131+
132+
#### Disabling Automatic Retry
133+
134+
To disable retries set the `:retry` option to `false`:
135+
136+
```rb
137+
gql = ShopifyAPI::GraphQL::Tiny.new(shop, token, :retry => false)
138+
```
139+
77140
### Pagination
78141

79142
In addition to built-in request retry `ShopifyAPI::GraphQL::Tiny` also builds in support for pagination.
@@ -196,19 +259,21 @@ pager.execute(query) { |page| }
196259

197260
The `"data"` and `"pageInfo"` keys are automatically added if not provided.
198261

199-
### Automatically Retrying Failed Requests
200-
201-
See [the docs](https://rubydoc.info/gems/shopify_api-graphql-tiny) for more information.
202-
203262
## Testing
204263

205264
`cp env.template .env` and fill-in `.env` with the missing values. This requires a Shopify store.
206265

266+
To elicit a request that will be rate-limited by Shopify run following Rake task:
267+
268+
```sh
269+
bundle exec rake rate_limit SHOPIFY_DOMAIN=your-domain SHOPIFY_TOKEN=your-token
270+
```
271+
207272
## See Also
208273

209274
- [Shopify Dev Tools](https://github.com/ScreenStaring/shopify-dev-tools) - Command-line program to assist with the development and/or maintenance of Shopify apps and stores
210-
- [Shopify ID Export](https://github.com/ScreenStaring/shopify_id_export/) Dump Shopify product and variant IDs —along with other identifiers— to a CSV or JSON file
211-
- [ShopifyAPIRetry](https://github.com/ScreenStaring/shopify_api_retry) - Retry a ShopifyAPI request if rate-limited or other errors occur (REST and GraphQL APIs)
275+
- [Shopify ID Export](https://github.com/ScreenStaring/shopify_id_export/) - Dump Shopify product and variant IDs —along with other identifiers— to a CSV or JSON file
276+
- [`TinyGID`](https://github.com/sshaw/tiny_gid/) - Build Global ID (gid://) URI strings from scalar values
212277

213278
## License
214279

‎Rakefile‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,161 @@ require "rspec/core/rake_task"
44
RSpec::Core::RakeTask.new(:spec)
55

66
task :default => :spec
7+
8+
desc "Elicit a Shopify rate limit"
9+
task :rate_limit do
10+
require "shopify_api/graphql/tiny"
11+
12+
query =<<-GQL
13+
query {
14+
products(first: 50, sortKey: UPDATED_AT, reverse: true) {
15+
pageInfo {
16+
hasNextPage
17+
endCursor
18+
}
19+
edges {
20+
node {
21+
id
22+
title
23+
handle
24+
status
25+
createdAt
26+
updatedAt
27+
publishedAt
28+
vendor
29+
productType
30+
tags
31+
descriptionHtml
32+
description
33+
onlineStoreUrl
34+
options(first: 3) {
35+
id
36+
name
37+
position
38+
values
39+
}
40+
variants(first: 80) {
41+
edges {
42+
node {
43+
id
44+
title
45+
price
46+
compareAtPrice
47+
sku
48+
barcode
49+
inventoryItem {
50+
id
51+
unitCost {
52+
amount
53+
currencyCode
54+
}
55+
countryCodeOfOrigin
56+
harmonizedSystemCode
57+
}
58+
inventoryPolicy
59+
taxable
60+
availableForSale
61+
metafields(first: 20) {
62+
edges {
63+
node {
64+
id
65+
namespace
66+
key
67+
value
68+
type
69+
description
70+
}
71+
}
72+
}
73+
}
74+
}
75+
}
76+
media(first: 20) {
77+
edges {
78+
node {
79+
__typename
80+
alt
81+
status
82+
... on MediaImage {
83+
id
84+
preview {
85+
image {
86+
url
87+
}
88+
}
89+
image {
90+
id
91+
url
92+
width
93+
height
94+
altText
95+
}
96+
}
97+
... on Video {
98+
id
99+
sources {
100+
url
101+
format
102+
height
103+
width
104+
}
105+
}
106+
... on ExternalVideo {
107+
id
108+
originUrl
109+
embedUrl
110+
}
111+
}
112+
}
113+
}
114+
images(first: 20) {
115+
edges {
116+
node {
117+
id
118+
url
119+
altText
120+
width
121+
height
122+
}
123+
}
124+
}
125+
seo {
126+
title
127+
description
128+
}
129+
metafields(first: 30) {
130+
edges {
131+
node {
132+
id
133+
namespace
134+
key
135+
value
136+
type
137+
ownerType
138+
}
139+
}
140+
}
141+
collections(first: 10) {
142+
edges {
143+
node {
144+
id
145+
title
146+
handle
147+
}
148+
}
149+
}
150+
}
151+
}
152+
}
153+
}
154+
GQL
155+
156+
threads = 5.times.map do
157+
Thread.new do
158+
gql = ShopifyAPI::GQL::Tiny.new(ENV.fetch("SHOPIFY_DOMAIN"), ENV.fetch("SHOPIFY_TOKEN"), :debug => true)
159+
pp gql.execute(query).dig("extensions", "cost", "throttleStatus")
160+
end
161+
end
162+
163+
threads.each(&:join)
164+
end

0 commit comments

Comments
 (0)