Object Lock
Last updated
Last updated
Object lock prevents object version deletion during a user-defined retention period. Immutable S3 objects are protected using object- or bucket-level configuration of WORM and retention attributes.
The retention policy is defined using the S3 API or bucket-level defaults.
Objects are locked for the duration of the retention period, and legal hold scenarios are also supported.
There are two lock types for object lock:
Retention period: Specifies a fixed period of time during which an object version remains locked. During this period, your object version is WORM-protected and can't be overwritten or deleted.
Legal hold: Provides the same protection as a retention period, but it has no expiration date. Instead, a legal hold remains in place until you explicitly remove it. Legal holds are independent from retention periods.
There are two modes for the retention period:
In Compliance mode, if you applied a wrong retention period (e.g 6 years instead of 6 days) cegedim.cloud has no possibility to delete or shorten the retention period.
A good practice is to start with Governance mode to perform tests, and then switch to Compliance mode.
For more information, see:
Object lock requires ADO (Access During Outage) disabled at the Object Store level
Object Stores without ADO cannot be created using ITCare and must therefore be created manually by cegedim.cloud teams
Object lock only works with IAM (not legacy accounts)
Object lock works only with versioned buckets
Enabling locking on the bucket automatically makes it versioned
Once bucket locking is enabled, it is not possible to disable object lock or suspend versioning for the bucket
Object lock requires FS (File System) disabled on bucket
Object lock is only supported by S3 API
A bucket has a default configuration including a retention mode (governance or compliance) and a retention period (which is days or years)
Object locks apply to individual object versions only
Different versions of a single object can have different retention modes and periods
A lock prevents an object from being deleted or overwritten. Overwritten does not mean that new versions can't be created (new versions can be created with their own lock settings)
An object can still be deleted version-wise. It creates a delete marker and the version still exists and is locked
Compliance mode is stricter : locks can't be removed, decreased, or downgraded to governance mode
Governance mode is less strict : locks can be removed, bypassed, or even elevated to compliance mode
Updating an object version's metadata, which occurs when you place or alter an object lock, doesn't overwrite the object version or reset its Last-Modified timestamp
Retention period can be placed on an object explicitly, or implicitly through a bucket default setting
Placing a default retention setting on a bucket doesn't place any retention settings on objects that already exist in the bucket
Changing a bucket's default retention period doesn't change the existing retention period for any objects in that bucket
Object lock and traditional bucket/object retention can co-exist
Objects under lock are protected from lifecycle deletions.
Lifecycle logic is made difficult because of the variety of behavior of different locks.
From a lifecycle point of view there are locks without a date, locks with date that can be extended, and locks with date that can be decreased.
For Compliance mode, the retain until date can't be decreased, but can be increased
For Governance mode, the lock date can increase, decrease, or be removed
For legal hold, the lock is indefinite
Access control using IAM policies is an important part of the object lock functionality.
The s3:BypassGovernanceRetention
permission is important because it is required to delete a WORM-protected object in Governance mode.
IAM policy conditions have been defined below to allow you to limit what retention period and legal hold can be specified in objects.
It is not possible to manage IAM Policies with ITCare.
s3:object-lock-legal-hold
Enables enforcement of the specified object legal hold status
s3:object-lock-mode
Enables enforcement of the specified object retention mode
s3:object-lock-retain-until-date
Enables enforcement of a specific retain-until-date
s3:object-lock-remaining-retention-days
Enables enforcement of an object relative to the remaining retention days
These condition keys can be used inside bucket and IAM policies to control object lock behaviors.
Example: ensure the retention days does not exceed 5 years
Create a bucket with object lock enabled:
aws s3api --endpoint-url=${S3_ENDPOINT} create-bucket --bucket bucket-with-lock --object-lock-enabled-for-bucket --profile ${S3_PROFILE}
{
"Location": "/bucket-with-lock"
}
You can only enable Object lock for new buckets.
You can't enable Object lock on an existing Bucket.
Add a lock configuration on your bucket:
{
"ObjectLockEnabled": "Enabled",
"Rule": {
"DefaultRetention": {
"Mode": "GOVERNANCE",
"Days": 1
}
}
}
aws s3api --endpoint-url=${S3_ENDPOINT} put-object-lock-configuration --bucket bucket-with-lock --object-lock-configuration file://governance.json --profile ${S3_PROFILE}
Get the current lock configuration on a bucket:
aws s3api --endpoint-url=${S3_ENDPOINT} get-object-lock-configuration --bucket bucket-with-lock --profile ${S3_PROFILE}
{
"ObjectLockConfiguration": {
"ObjectLockEnabled": "Enabled",
"Rule": {
"DefaultRetention": {
"Mode": "GOVERNANCE",
"Days": 1
}
}
}
}
In this context, there is no lock configuration defined at the bucket level but the bucket was created with --object-lock-enabled-for-bucket
.
Put an object into a bucket:
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 cp feather.ttf s3://bucket-test
upload: ./feather.ttf to s3://bucket-test/feather.ttf
Apply retention period on the object :
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api put-object-retention --retention Mode=GOVERNANCE,RetainUntilDate=2022-08-26T17:00:00 --bucket bucket-test --key feather.ttf
Use head-object
method to get object metadata and get retention information:
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api head-object --bucket bucket-test --key feather.ttf
{
"LastModified": "Thu, 25 Aug 2022 12:32:09 GMT",
"ContentLength": 81512,
"ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
"VersionId": "1661430729953",
"ContentType": "font/ttf",
"ServerSideEncryption": "AES256",
"Metadata": {},
"ObjectLockMode": "GOVERNANCE",
"ObjectLockRetainUntilDate": "2022-08-26T17:00:00.000Z"
}
Increase retention (+1 days):
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api put-object-retention --retention Mode=GOVERNANCE,RetainUntilDate=2022-08-27T17:00:00 --bucket bucket-test --key feather.ttf
{
"LastModified": "Thu, 25 Aug 2022 12:32:09 GMT",
"ContentLength": 81512,
"ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
"VersionId": "1661430729953",
"ContentType": "font/ttf",
"ServerSideEncryption": "AES256",
"Metadata": {},
"ObjectLockMode": "GOVERNANCE",
"ObjectLockRetainUntilDate": "2022-08-27T17:00:00.000Z"
}
Delete the object:
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 rm s3://bucket-test/feather.ttf
delete: s3://bucket-test/feather.ttf
# List the content of the bucket, the feather.ttf is not display anymore
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/
# No Output
Check version:
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf
{
"Versions": [
{
"ETag": "\"1747b668712195f92c827c7b23a169fc\"",
"Size": 5,
"StorageClass": "STANDARD",
"Key": "feather.ttf",
"VersionId": "1661431782902",
"IsLatest": false,
"LastModified": "2022-08-25T12:49:42.902Z",
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
}
}
],
"DeleteMarkers": [
{
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud"
},
"Key": "feather.ttf",
"VersionId": "1661431795694",
"IsLatest": true,
"LastModified": "2022-08-25T12:49:55.694Z"
}
]
}
Delete the delete marker using the version Id:
aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661431795694
{
"DeleteMarker": true,
"VersionId": "1661431795694"
}
# Once delete marker deleted, the object can be listed again
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/feather.ttf
2022-08-25 14:49:42 5 feather.ttf
Upload a new version of the object:
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf
{
"Versions": [
{
"ETag": "\"2232dadea2f05fa28e3f08b5b3346df9\"",
"Size": 81512,
"StorageClass": "STANDARD",
"Key": "feather.ttf",
"VersionId": "1661432092692",
"IsLatest": true,
"LastModified": "2022-08-25T12:54:52.692Z",
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
}
},
{
"ETag": "\"1747b668712195f92c827c7b23a169fc\"",
"Size": 5,
"StorageClass": "STANDARD",
"Key": "feather.ttf",
"VersionId": "1661431782902",
"IsLatest": false,
"LastModified": "2022-08-25T12:49:55.694Z",
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
}
}
]
}
Delete a specific version:
aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661432092692
{
"VersionId": "1661432092692"
}
Lister les versions :
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf
{
"Versions": [
{
"ETag": "\"1747b668712195f92c827c7b23a169fc\"",
"Size": 5,
"StorageClass": "STANDARD",
"Key": "feather.ttf",
"VersionId": "1661431782902",
"IsLatest": false,
"LastModified": "2022-08-25T12:49:55.694Z",
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
}
}
],
"DeleteMarkers": [
{
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:user/cloud"
},
"Key": "feather.ttf",
"VersionId": "1661432197362",
"IsLatest": true,
"LastModified": "2022-08-25T12:56:37.362Z"
}
]
}
Use the --bypass-governance-retention
option to bypass the governance policy and delete the delete marker:
aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661432197362 --bypass-governance-retention
{
"DeleteMarker": true,
"VersionId": "1661432197362"
}
List versions:
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf
{
"Versions": [
{
"ETag": "\"1747b668712195f92c827c7b23a169fc\"",
"Size": 5,
"StorageClass": "STANDARD",
"Key": "feather.ttf",
"VersionId": "1661431782902",
"IsLatest": true,
"LastModified": "2022-08-25T12:49:55.694Z",
"Owner": {
"DisplayName": "urn:ecs:iam::cos-cegedimit-test-lock:root",
"ID": "urn:ecs:iam::cos-cegedimit-test-lock:root"
}
}
]
}
Delete the latest version of the object with --bypass-governance-retention:
aws --profile=${S3_PROFILE} --endpoint ${S3_ENDPOINT} s3api delete-object --bucket bucket-test --key feather.ttf --version-id 1661431782902 --bypass-governance-retention
{
"VersionId": "1661431782902"
}
List versions:
aws --profile=${PROFILE} --endpoint ${S3_ENDPOINT} s3api list-object-versions --bucket bucket-test --prefix feather.ttf
Empty result.
List bucket content:
aws --endpoint-url=${S3_ENDPOINT} --profile=${S3_PROFILE} s3 ls s3://bucket-test/
Empty bucket.
Found example of lock configuration that can be applied on a bucket.
The retention will be applied on each object put in the bucket.
The default Object Lock retention mode you want to apply to new objects placed in the specified bucket. Must be used with either Days or Years.
Possible values are COMPLIANCE or GOVERNANCE.
The number of days that you want to specify for the default retention period. Must be used with Mode.
Years
The number of years that you want to specify for the default retention period. Must be used with Mode.
Days and Years are mutually exclusive
S3 Browser is a freeware Windows client for Amazon S3.
With S3 Browser you can see headers on objects and get the current lock retention applied on the object:
That means without ADO, Object operations of read, create, update and delete as well as list buckets not owned by an online site, will fail
IAM accounts are not managed by ITCare and must therefore be created manually by cegedim.cloud teams