Object Lock

Object lock prevents object version deletion during a user-defined retention period. Immutable S3 objects are protected using object- or bucket-level configuration of WORM and retention attributes.

The retention policy is defined using the S3 API or bucket-level defaults.

Objects are locked for the duration of the retention period, and legal hold scenarios are also supported.

There are two lock types for object lock:

• Retention period: Specifies a fixed period of time during which an object version remains locked. During this period, your object version is WORM-protected and can't be overwritten or deleted.

• Legal hold: Provides the same protection as a retention period, but it has no expiration date. Instead, a legal hold remains in place until you explicitly remove it. Legal holds are independent from retention periods.

There are two modes for the retention period:

Governance mode

Users can't overwrite or delete an object version or alter its lock settings unless they have special permissions.

With Governance mode, you protect objects against being deleted by most users, but you can still grant some users permission to alter the retention settings or delete the object if necessary.

You can also use Governance mode to test retention-period settings before creating a compliance-mode retention period.

• Users cannot overwrite or delete an object version.

• Users with s3:PutObjectRetention permission can increase an object retention period.

• Users with special s3:BypassGovernanceRetention permission can remove or shorten an object retention.

• Users with s3:BypassGovernanceRetention permission can also delete locked objects.

Compliance mode

A protected object version can't be overwritten or deleted by any user, including the root user in your account.

When an object is locked in compliance mode, its retention mode can't be changed, and its retention period can't be shortened.

Compliance mode helps ensure that an object version can't be overwritten or deleted for the duration of the retention period.

• Users cannot overwrite or delete an object version.

• Users with s3:PutObjectRetention permission can increase an object retention period.

• User cannot remove or shorten an object retention.

In Compliance mode, if you applied a wrong retention period (e.g 6 years instead of 6 days) cegedim.cloud has no possibility to delete or shorten the retention period.

A good practice is to start with Governance mode to perform tests, and then switch to Compliance mode.

For more information, see:

Protecting data with Amazon S3 Object Lock | Amazon Web ServicesAmazon Web Services

Object Lock Requirements

• Object lock requires ADO (Access During Outage) disabled at the Object Store level

  • That means without ADO, Object operations of read, create, update and delete as well as list buckets not owned by an online site, will fail

  • Object Stores without ADO cannot be created using ITCare and must therefore be created manually by cegedim.cloud teams

• Object lock only works with IAM (not legacy accounts)

• IAM accounts are not managed by ITCare and must therefore be created manually by cegedim.cloud teams

• Object lock works only with versioned buckets

• Enabling locking on the bucket automatically makes it versioned

• Once bucket locking is enabled, it is not possible to disable object lock or suspend versioning for the bucket

• Object lock requires FS (File System) disabled on bucket

• Object lock is only supported by S3 API

• A bucket has a default configuration including a retention mode (governance or compliance) and a retention period (which is days or years)

• Object locks apply to individual object versions only

• Different versions of a single object can have different retention modes and periods

• A lock prevents an object from being deleted or overwritten. Overwritten does not mean that new versions can't be created (new versions can be created with their own lock settings)

• An object can still be deleted version-wise. It creates a delete marker and the version still exists and is locked

• Compliance mode is stricter : locks can't be removed, decreased, or downgraded to governance mode

• Governance mode is less strict : locks can be removed, bypassed, or even elevated to compliance mode

• Updating an object version's metadata, which occurs when you place or alter an object lock, doesn't overwrite the object version or reset its Last-Modified timestamp

• Retention period can be placed on an object explicitly, or implicitly through a bucket default setting

• Placing a default retention setting on a bucket doesn't place any retention settings on objects that already exist in the bucket

• Changing a bucket's default retention period doesn't change the existing retention period for any objects in that bucket

• Object lock and traditional bucket/object retention can co-exist

Lifecycle

Objects under lock are protected from lifecycle deletions.

Lifecycle logic is made difficult because of the variety of behavior of different locks.

From a lifecycle point of view there are locks without a date, locks with date that can be extended, and locks with date that can be decreased.

• For Compliance mode, the retain until date can't be decreased, but can be increased

• For Governance mode, the lock date can increase, decrease, or be removed

• For legal hold, the lock is indefinite

Condition Keys

Access control using IAM policies is an important part of the object lock functionality.

The s3:BypassGovernanceRetention permission is important because it is required to delete a WORM-protected object in Governance mode.

IAM policy conditions have been defined below to allow you to limit what retention period and legal hold can be specified in objects.

It is not possible to manage IAM Policies with ITCare.

Condition key	Description
s3:object-lock-legal-hold	Enables enforcement of the specified object legal hold status
s3:object-lock-mode	Enables enforcement of the specified object retention mode
s3:object-lock-retain-until-date	Enables enforcement of a specific retain-until-date
s3:object-lock-remaining-retention-days	Enables enforcement of an object relative to the remaining retention days

These condition keys can be used inside bucket and IAM policies to control object lock behaviors.

Example: ensure the retention days does not exceed 5 years

Examples and use cases

Buckets

We use aws s3 and aws s3api command line tools from AWSCLIv2 on Linux. ${S3_ENDPOINT} and ${S3_PROFILE} are environment variables.

Create a bucket with object lock enabled:

aws s3api --endpoint-url=${S3_ENDPOINT} create-bucket --bucket bucket-with-lock --object-lock-enabled-for-bucket --profile ${S3_PROFILE}

Output

{
    "Location": "/bucket-with-lock"
}

You can only enable Object lock for new buckets.

You can't enable Object lock on an existing Bucket.

Add a lock configuration on your bucket:

governance.json

{
  "ObjectLockEnabled": "Enabled",
  "Rule": {
    "DefaultRetention": {
      "Mode": "GOVERNANCE",
      "Days": 1
    }
  }
}

aws s3api --endpoint-url=${S3_ENDPOINT} put-object-lock-configuration --bucket bucket-with-lock --object-lock-configuration file://governance.json --profile ${S3_PROFILE}

Get the current lock configuration on a bucket:

aws s3api --endpoint-url=${S3_ENDPOINT} get-object-lock-configuration --bucket bucket-with-lock --profile ${S3_PROFILE}

Output

{
    "ObjectLockConfiguration": {
        "ObjectLockEnabled": "Enabled",
        "Rule": {
            "DefaultRetention": {
                "Mode": "GOVERNANCE",
                "Days": 1
            }
        }
    }
}

Objects

In this context, there is no lock configuration defined at the bucket level but the bucket was created with --object-lock-enabled-for-bucket.

We use aws s3 and aws s3api command line tools from AWSCLIv2 on Linux. ${S3_ENDPOINT} and ${S3_PROFILE} are environment variables.

Put an object into a bucket:

aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 cp feather.ttf s3://bucket-test

Output

upload: ./feather.ttf to s3://bucket-test/feather.ttf

Apply retention period on the object :

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api put-object-retention --retention Mode=GOVERNANCE,RetainUntilDate=2022-08-26T17:00:00 --bucket bucket-test --key feather.ttf

Use head-object method to get object metadata and get retention information:

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api head-object --bucket bucket-test --key feather.ttf

Output

{
    "LastModified": "Thu, 25 Aug 2022 12:32:09 GMT",
    "ContentLength": 81512,
    "ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
    "VersionId": "1661430729953",
    "ContentType": "font/ttf",
    "ServerSideEncryption": "AES256",
    "Metadata": {},
    "ObjectLockMode": "GOVERNANCE",
    "ObjectLockRetainUntilDate": "2022-08-26T17:00:00.000Z"
}

Increase retention (+1 days):

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api put-object-retention --retention Mode=GOVERNANCE,RetainUntilDate=2022-08-27T17:00:00 --bucket bucket-test --key feather.ttf

Output

{
    "LastModified": "Thu, 25 Aug 2022 12:32:09 GMT",
    "ContentLength": 81512,
    "ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
    "VersionId": "1661430729953",
    "ContentType": "font/ttf",
    "ServerSideEncryption": "AES256",
    "Metadata": {},
    "ObjectLockMode": "GOVERNANCE",
    "ObjectLockRetainUntilDate": "2022-08-27T17:00:00.000Z"
}

Delete the object:

aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 rm s3://bucket-test/feather.ttf

Output

delete: s3://bucket-test/feather.ttf

# List the content of the bucket, the feather.ttf is not display anymore
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/
 
# No Output

Check version:

This will display all object's versions as well as the delete marker, created when we delete the object, previously.

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf

Output

{
    "Versions": [
        {
            "ETag": "\"1747b668712195f92c827c7b23a169fc\"",
            "Size": 5,
            "StorageClass": "STANDARD",
            "Key": "feather.ttf",
            "VersionId": "1661431782902",
            "IsLatest": false,
            "LastModified": "2022-08-25T12:49:42.902Z",
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
            }
        }
    ],
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud"
            },
            "Key": "feather.ttf",
            "VersionId": "1661431795694",
            "IsLatest": true,
            "LastModified": "2022-08-25T12:49:55.694Z"
        }
    ]
}

Delete the delete marker using the version Id:

aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661431795694

Output

{
    "DeleteMarker": true,
    "VersionId": "1661431795694"
}

# Once delete marker deleted, the object can be listed again
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/feather.ttf

Output

2022-08-25 14:49:42          5 feather.ttf

Upload a new version of the object:

feather.ttf object now has two versions.

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf

Output

{
    "Versions": [
        {
            "ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
            "Size": 81512,
            "StorageClass": "STANDARD",
            "Key": "feather.ttf",
            "VersionId": "1661432092692",
            "IsLatest": true,
            "LastModified": "2022-08-25T12:54:52.692Z",
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
            }
        },
        {
            "ETag": "\"1747b668712195f92c827c7b23a169fc\"",
            "Size": 5,
            "StorageClass": "STANDARD",
            "Key": "feather.ttf",
            "VersionId": "1661431782902",
            "IsLatest": false,
            "LastModified": "2022-08-25T12:49:55.694Z",
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
            }
        }
    ]
}

Delete a specific version:

The deleted version "becomes" a delete marker.

aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661432092692

Output

{
    "VersionId": "1661432092692"
}

Lister les versions :

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf

Output

{
    "Versions": [
        {
            "ETag": "\"1747b668712195f92c827c7b23a169fc\"",
            "Size": 5,
            "StorageClass": "STANDARD",
            "Key": "feather.ttf",
            "VersionId": "1661431782902",
            "IsLatest": false,
            "LastModified": "2022-08-25T12:49:55.694Z",
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
            }
        }
    ],
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud"
            },
            "Key": "feather.ttf",
            "VersionId": "1661432197362",
            "IsLatest": true,
            "LastModified": "2022-08-25T12:56:37.362Z"
        }
    ]
}

Use the --bypass-governance-retention option to bypass the governance policy and delete the delete marker:

aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661432197362 --bypass-governance-retention

Output

{
    "DeleteMarker": true,
    "VersionId": "1661432197362"
}

List versions:

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf

Output

{
    "Versions": [
        {
            "ETag": "\"1747b668712195f92c827c7b23a169fc\"",
            "Size": 5,
            "StorageClass": "STANDARD",
            "Key": "feather.ttf",
            "VersionId": "1661431782902",
            "IsLatest": true,
            "LastModified": "2022-08-25T12:49:55.694Z",
            "Owner": {
                "DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
                "ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
            }
        }
    ]
}

Delete the latest version of the object with --bypass-governance-retention:

aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661431782902 --bypass-governance-retention

Output

{
    "VersionId": "1661431782902"
}

List versions:

aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf

Empty result.

List bucket content:

aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/

Empty bucket.

Object Lock configuration

Found example of lock configuration that can be applied on a bucket.

The retention will be applied on each object put in the bucket.

Lock Configuration Structure.

Lock configuration is a JSON document:

{
  "ObjectLockEnabled": "Enabled",
  "Rule": {
    "DefaultRetention": {
      "Mode": "GOVERNANCE"|"COMPLIANCE",
      "Days": integer,
      "Years": integer
    }
  }
}

Mode

The default Object Lock retention mode you want to apply to new objects placed in the specified bucket. Must be used with either Days or Years.

Possible values are COMPLIANCE or GOVERNANCE.

Days

The number of days that you want to specify for the default retention period. Must be used with Mode.

Years

The number of years that you want to specify for the default retention period. Must be used with Mode.

Days and Years are mutually exclusive

Governance configuration

{
    "ObjectLockConfiguration": {
        "ObjectLockEnabled": "Enabled",
        "Rule": {
            "DefaultRetention": {
                "Mode": "GOVERNANCE",
                "Years": 1
            }
        }
    }
}

Compliance configuration

{
    "ObjectLockConfiguration": {
        "ObjectLockEnabled": "Enabled",
        "Rule": {
            "DefaultRetention": {
                "Mode": "COMPLIANCE",
                "Days": 60
            }
        }
    }
}

S3 Browser

S3 Browser is a freeware Windows client for Amazon S3.

S3 Browser doesn't allow you to manage locks on buckets or objects.

With S3 Browser you can see headers on objects and get the current lock retention applied on the object: