Stubbing and allowing HTTP requests from within the sandboxer

By default, the Grok sandbox does not allow any network requests to be made. As a content author, you have some control over the handing of HTTP and HTTPS requests made within the sandbox. This is controlled at the per-workspace level, which also implies per-testcase.

The special ___http_config.json file

When a workspace contains an encrypted file called `___http_config.json`, Grok interprets this file as the configuration for the transparent proxy. This allows you to stub HTTP(S) requests, as well as expose certain URL patterns out to the real internet (with limitations).

The grammar for this special JSON file is as follows:

___http_config.json:
{
  OPTIONAL "expose": Expose
  OPTIONAL "stubs": [Stub]
}

Expose:
{
  "requests": [ExposeRequest]
  OPTIONAL "limit": integer limit on the number of expose requests that can be made. There is a default limit of 20.
}

ExposeRequest:
{
  RequestCondition
  OPTIONAL "limit": integer limit on the number of requests that can be made. There is a default limit of 20.
}

Stub:
{
  "request": RequestCondition
  "response": StubResponse
}

RequestCondition:
{
  "method": one of "GET", "POST", "PATCH", "PUT", "DELETE", "OPTIONS", "HEAD".
  "url": a regex string of the full URL, including protocol, e.g. "https?://my-domain.com/api/(weather|chickens)\.json".
}

StubResponse:
{
  "status": integer HTTP status
  "content": UTF-8 string content value
  OPTIONAL "headers": {
    string key: string value or list of string values
  },
}

This allows you to both stub responses to requests, and expose certain requests. We strongly recommend stubbing over exposing wherever possible.

For example, here's an example of a workspace that has an entirely stubbed fake API, which has responses for user IDs 1 and 2, and 404's for all other IDs:

#markdown
```lang:py3;
import requests

user_id = int(input('Enter user ID: '))
response = requests.get(f'https://some-random-api.com/api/user/{user_id}/')
print(response.status_code)
print(response.text)
```

```eg:last;path:___http_config.json;encrypted;exportable;
{
  "stubs": [
    {
      "request": {
        "method": "GET",
        "url": "^https://some-random-api\\.com/api/user/1/$"
      },
      "response": {
        "status": 200,
        "content": "{\"id\": 1, \"name\": \"Jane Doe\"}",
        "headers": {
          "content-type": "application/json"
        }
      }
    },
    {
      "request": {
        "method": "GET",
        "url": "^https://some-random-api\\.com/api/user/2/$"
      },
      "response": {
        "status": 200,
        "content": "{\"id\": 2, \"name\": \"John Smith\"}",
        "headers": {
          "content-type": "application/json"
        }
      }
    },
    {
      "request": {
        "method": "GET",
        "url": "^https://some-random-api\\.com/api/user/[0-9]/$"
      },
      "response": {
        "status": 404,
        "content": "{\"error\": \"User not found.\"}",
        "headers": {
          "content-type": "application/json"
        }
      }
    }
  ]
}
```