AirMettle Select lets you run SQL queries directly against CSV, JSON, and JSON Lines objects in object storage. This quickstart walks through a typical workflow and shows how to get started quickly.
1. Pick a blob to query
Pick a CSV or JSON blob in your object storage account that you want to query. You need the object URL plus the account name and key. AirMettle Select runs in Azure East US 2 today, so a storage account in that region is a good place to start.
2. Install the Python package
Use the Python client in this quickstart and install httpx for API calls.
pip install pyamselect httpx
3. Save your credentials
Add your subscription ID, API key, and storage account key as environment variables so credentials stay out of source code.
Linux / macOS / WSL / Git Bash
export AIRMETTLE_SUBSCRIPTION_ID="YOUR_SUBSCRIPTION_ID"
export AIRMETTLE_API_KEY="YOUR_API_KEY"
export STORAGE_ACCOUNT_KEY="YOUR_STORAGE_ACCOUNT_KEY"
PowerShell
$env:AIRMETTLE_SUBSCRIPTION_ID="YOUR_SUBSCRIPTION_ID"
$env:AIRMETTLE_API_KEY="YOUR_API_KEY"
$env:STORAGE_ACCOUNT_KEY="YOUR_STORAGE_ACCOUNT_KEY"
Command Prompt
set AIRMETTLE_SUBSCRIPTION_ID=YOUR_SUBSCRIPTION_ID
set AIRMETTLE_API_KEY=YOUR_API_KEY
set STORAGE_ACCOUNT_KEY=YOUR_STORAGE_ACCOUNT_KEY
4. Link your storage account using the API
Link your object storage account to AirMettle Select. This is a one-time step per account.
import os
import httpx
# Include your subscription credentials in the request headers
headers = {
"x-subscription-id": os.getenv("AIRMETTLE_SUBSCRIPTION_ID"),
"x-api-key": os.getenv("AIRMETTLE_API_KEY"),
}
# Create the request body with the storage account name and key
request = {
"storage_account": "yourstorageaccount",
"account_key": os.getenv("STORAGE_ACCOUNT_KEY")
}
response = httpx.post("https://api.airmettle.com/api/v1/buckets", json=request, headers=headers)
print(response.status_code, response.text)
A 201 Created response means the account was linked successfully.
5. Create metadata for the blob using the API
Before querying a blob, AirMettle Select generates a small sidecar metadata file. These sidecars live in a hidden directory alongside your data and enable parallel high-performance reads.
You can generate a sidecar on demand. The original blob is never modified, and you can delete and regenerate the sidecar at any time.
import os
import httpx
# Specify the blob URL and your subscription credentials in the request body
request = {
"subscription_id": os.getenv("AIRMETTLE_SUBSCRIPTION_ID"),
"api_key": os.getenv("AIRMETTLE_API_KEY"),
"blob_url": "https://yourstore.blob.core.windows.net/container/yourblob.csv",
"account_key": os.getenv("STORAGE_ACCOUNT_KEY")
}
response = httpx.post("https://api.airmettle.com/v1/prepare", json=request)
print(response.status_code, response.text)
A 200 OK response means the metadata object is ready and queryable. A 202 Accepted response means generation is asynchronous and includes a status_url for polling.
Status values:
ready means the sidecar is available.
pending means generation is still in progress.
failed means metadata generation failed; review error details in the response body.
6. Run a query
In step 5, you prepared a blob using its full blob_url. In this step, identify the same blob by storage_account, container, and blob. The response is a stream of selected rows in the requested format.
import os
from pyamselect import AMSelectClient, SelectRequest, CSVInputOptions, CSVOutputOptions
# Specify which blob to query, the SQL, and input/output options
request = SelectRequest(
subscription_id=os.getenv("AIRMETTLE_SUBSCRIPTION_ID"),
api_key=os.getenv("AIRMETTLE_API_KEY"),
storage_account="yourstorageaccount",
container="yourcontainer",
blob="yourblob.csv",
expression="SELECT * FROM s3object", # Your SQL query
input_options=CSVInputOptions(csv_header_config="use",csv_field_delimiter=","), # input as CSV
output_options=CSVOutputOptions(), # output as CSV
)
# Create the client to connect to AirMettle Select
with AMSelectClient("api.airmettle.com/query", 8443) as client:
# Run the query and collect the results as a string
result = client.select_to_string(request)
print(result)
# Alternatively, write results directly to a file
# client.select_to_file(request, "output.csv")
# Or write results to a binary stream, such as a BytesIO object
# client.select_to_stream(request, stream)
