S3 API Compatibility

The Intercolo Object Storage service uses an S3 compatible API to interact with the storage backend.

This has the advantage of not needing to learn a new API or develop new tooling. In most cases, getting started is as simple as switching over the backend url and credentials to the ones created on the Api Keys page.

Overview

The main differences are:

  • Anonymous access is not allowed, all requests need to be authenticated (signed urls are supported). We do offer a CDN service that works in tandem with our object storage for anonymous access to files.

Authentication

The S3-Compatible API only supports signature version 4. Signature versions 2 and 4A are not supported.

Data Consistency

Interactions with the intercolo object storage are generally consistent and atomic. Once an api call returns, api calls after this one will reflect any change immediately. Any meta-data read will also never be partial or corrupt. This follows the strong read-after-write consistency model of Amazon S3.

The only exception to this consistency is downloading object contents for large multipart objects. We immediately queue data for deletion once an object is deleted or overwritten, so in-progress reads may abort early when they try reading from a part that does not exist anymore. Generally, we have found this to be a non-issue and an acceptable trade-off for data security, if this should be an issue for your use case please do get in touch, and we'll be happy to work on a solution with you.

Supported Actions

The S3-Compatible API currently supports (fully or partially) the following actions:

Differences compared to AWS S3

Concepts

Locations

Currently our one and only location for object storage is Frankfurt in Germany which has the location code: de-fra

Anonymous access

Anonymous access is not allowed, all requests need to be authenticated (signed urls are supported). We do offer a CDN service that works in tandem with our object storage for anonymous access to files.

Requester Pays

All requests to our S3 compatible api can only be accessed by the owner or via an url signed by the owner. The x-amz-request-payer request header is ignored/does not apply and accordingly there will not be a x-amz-request-charged header in responses.

Encryption

We currently do not support explicitly configuring encryption. This includes Managed Encryption Keys SSE-S3, KMS Keys (SSE-KMS), Dual-layer server-side encryption (DSEE-KMS) and customer-provided encryption keys (SSE-C). This also means any encryption related headers will be ignored/not sent in response

Request headers:

  • x-amz-server-side-encryption-customer-algorithm
  • x-amz-server-side-encryption-customer-key
  • x-amz-server-side-encryption-customer-key-MD5

Response headers:

  • x-amz-server-side-encryption
  • x-amz-server-side-encryption-customer-algorithm
  • x-amz-server-side-encryption-customer-key-MD5
  • x-amz-server-side-encryption-aws-kms-key-id
  • x-amz-server-side-encryption-bucket-key-enabled

Object Locks

Object locks are currently not supported. You can, however, configure an API key to be unable to overwrite or delete objects.

ACLs

Per object ACLs are not supported and permissions can be managed outside the S3 API per bucket per api key.

Lifecycle / Expiring objects

Expiring objects are currently not supported.

Versioning

Versioning is not supported.

Replication

Replication is configured per bucket at bucket creation time and happens transparently at upload time. Replications default to 1 and is currently limited to 3 replications max. This is easily done in the Intercolo Dashboard, but can also be achieved via the S3 API by specifying the number of replications after the location separated by a colon So, for example, for the default number of replications (1) one would simply pass de-fra as the location code. For two replications, de-fra:2 would be passed. For three replications, de-fra:3 would be passed.

Storage classes

There is currently only a standard storage class. Higher reliability and durability can be achieved with replication.

Object Tagging

Object tagging is not supported.

Website Hosting

Hosting websites directly on the object storage is not supported. The x-amz-website-redirect-location header will be ignored.

x-amz-expected-bucket-owner Header

The x-amz-expected-bucket-owner and x-amz-source-expected-bucket-owner headers are unsupported and will be ignored.

Per-Action Differences

Any limitations specific to actions other than the general above-mentioned ones

HeadBucket

No known additional differences.

CreateBucket

Only the LocationConstraint parameter in the CreateBucketConfiguration object is supported. Everything else is ignored. Unsupported headers:

  • x-amz-acl
  • x-amz-grant-full-control
  • x-amz-grant-read
  • x-amz-grant-read-acp
  • x-amz-grant-write
  • x-amz-grant-write-acp
  • x-amz-bucket-object-lock-enabled
  • x-amz-object-ownership

DeleteBucket

No known additional differences.

GetBucketLocation

No known additional differences.

GetObjectLockConfiguration

No known additional differences.

GetBucketVersioning

No known additional differences.

ListBuckets

No known additional differences.

ListObjects / ListObjectsV2

Unsupported headers:

  • x-amz-optional-object-attributes is currently not respected and ignored

HeadObject

Unsupported headers:

  • x-amz-checksum-mode Is currently not respected, checksums are always returned
  • If-Match, If-Modified-Since, If-None-Match and If-Unmodified-Since are currently not respected, but are planned to be implemented

GetObject

Unsupported headers:

  • x-amz-checksum-mode Is currently not respected, checksums are always returned
  • If-Match, If-Modified-Since, If-None-Match and If-Unmodified-Since are currently not respected, but are planned to be implemented

Overriding response headers by providing their respective values in the request prefixes with response- is currently not supported. These include:

  • Response-Content-Type
  • Response-Content-Language
  • Response-Expires
  • Response-Cache-Control
  • Response-Content-Disposition
  • Response-Content-Encoding

CopyObject

Unsupported Headers:

  • x-amz-copy-source-if-match is currently unsupported & ignored
  • x-amz-copy-source-if-none-match is currently unsupported & ignored
  • x-amz-copy-source-if-unmodified-since is currently unsupported & ignored
  • x-amz-copy-source-if-modified-since is currently unsupported & ignored
  • x-amz-checksum-algorithm is ignored, all checksums are computed

PutObject

Unsupported Headers (in addition to general limitations, see above):

  • x-amz-sdk-checksum-algorithm is unsupported & ignored
  • Expires is unsupported & ignored

DeleteObject

Deleted bucket names will not be made available for reuse immediately.

DeleteObjects

Unsupported Headers:

  • x-amz-sdk-checksum-algorithm is unsupported & ignored

ListMultipartUploads

No known additional differences.

CreateMultipartUpload

Unsupported Headers:

  • x-amz-checksum-algorithm is ignored, all checksums are computed

AbortMultipartUpload

No known additional differences.

CompleteMultipartUpload

No known additional differences.

UploadPart

No known additional differences.

UploadPartCopy

Unsupported Headers:

  • x-amz-copy-source-if-match is currently unsupported & ignored
  • x-amz-copy-source-if-none-match is currently unsupported & ignored
  • x-amz-copy-source-if-unmodified-since is currently unsupported & ignored
  • x-amz-copy-source-if-modified-since is currently unsupported & ignored