Bucket Lifecycle

The lifecycle configuration allow you to set an expiration policy on your objects and auto-delete them.

For example, you may need for some objects to be deleted automatically.

In this example, we will automatically create a policy to delete objects with a key starting with reports/ after 90 days. We could use a GUI Client or AWS SDK, but we will use AWS CLI to do so.

Limitations

• Lifecycle is a bucket level concept.

• Maximum of 1000 lifecycle rules per bucket is applicable.

• There may be a delay between the expiration date and the date at which Object Storage Service removes an object.

• Always round up the resulting time to the next day midnight UTC.

Deleted Object cannot be restored.

Manage lifecycle policy

Bucket lifecycle configuration can be managed using aws s3api (other tools or SDK works too):

• put-bucket-lifecycle

• get-bucket-lifecycle

• delete-bucket-lifecycle

We use aws s3 and aws s3apicommand line tools from AWSCLIv2 on Linux.

${S3_ENDPOINT} & ${S3_PROFILE} are environment variables.

Create a lifecycle policy

Create JSON file and put your policy in it:

delete_after_3_days.json

{
  "Rules": [
    {
      "Filter": {
        "Prefix": ""
      },
      "Expiration": {
        "Days": 3
      },
      "Status": "Enabled",
      "ID": "Delete After 3 days."
    }
  ]
}

Apply it to the bucket: bucket-test

aws s3api --endpoint-url=${S3_ENDPOINT} put-bucket-lifecycle --bucket bucket-test --lifecycle-configuration file://delete_after_3days.json --profile ${S3_PROFILE}

Get a lifecycle configuration

aws s3api --endpoint-url=${S3_ENDPOINT} get-bucket-lifecycle --bucket bucket-test --profile ${S3_PROFILE}

Delete a lifecycle configuration

aws s3api --endpoint-url=${S3_ENDPOINT} delete-bucket-lifecycle --bucket bucket-test --profile ${S3_PROFILE}

Supported lifecycle configuration elements

Name	Description	Required
And	Container for specify rule filters. These filters determine the subset of objects to which the rule applies. Type: String Ancestor: Rule	Yes, if you specify more than one filter condition (for example, one prefix and one or more tags).
Date	Date when you want S3 to take the action. The date value must conform to the ISO 8601 format. The time is always midnight UTC. Type: String Ancestor: Expiration	Yes, if Days and ExpiredObjectDeleteMarker are absent.
Days	Specifies the number of days after object creation when the specific rule action takes effect. Type: Nonnegative Integer when used with Transition, Positive Integer when used with Expiration. Ancestor: Expiration	Yes, if Date and ExpiredObjectDeleteMarker are absent.
Expiration	This action specifies a period in an object's lifetime when S3 should take the appropriate expiration action. The action S3 takes depends on whether the bucket is versioning-enabled. If versioning has never been enabled on the bucket, S3 deletes the only copy of the object permanently. Otherwise, if your bucket is versioning-enabled (or versioning is suspended), the action applies only to the current version of the object. A versioning-enabled bucket can have many versions of the same object, one current version, and zero or more non-current versions. Instead of deleting the current version, S3 makes it a non-current version by adding a delete marker as the new current version. Note: If your bucket state is versioning-suspended, S3 creates a delete marker with version ID null. If you have a version with version ID null, then S3 overwrites that version. To set expiration for non-current objects, you must use the NoncurrentVersionExpiration action. Type: Container Children: Days or Date Ancestor: Rule	Yes, if no other action is present in the Rule.
Filter	Container for elements that describe the filter identifying a subset of objects to which the lifecycle rule applies. If you specify an empty filter ("Prefix": {}), the rule applies to all objects in the bucket. Type: String Children: Prefix, Tag Ancestor: Rule	Yes
ID	Unique identifier for the rule. The value cannot be longer than 255 characters. Type: String Ancestor: Rule	No
Key	Specifies the key of a tag. A tag key can be up to 128 Unicode characters in length. Tag keys that you specify in a lifecycle rule filter must be unique. Type: String Ancestor: Tag	Yes, if <Tag> parent is specified.
LifecycleConfiguration	Container for lifecycle rules. You can add as many as 1,000 rules. Type: Container Children: Rule Ancestor: None	Yes
ExpiredObjectDeleteMarker	On a versioned bucket (versioning-enabled or versioning-suspended bucket), you can add this element in the lifecycle configuration to direct S3 to delete expired object delete markers. On a non-versioned bucket, adding this element in a policy is meaningless because you cannot have delete markers and the element does not do anything. When you specify this lifecycle action, the rule cannot specify a tag-based filter. Type: String Valid values: true | false (the value false is allowed, but it is no-op and S3 does not take action if the value is false) Ancestor: Expiration.	Yes, if Date and Days are absent.
NoncurrentDays	Specifies the number of days an object is non-current before S3 can perform the associated action. Type: Positive Integer when used with NoncurrentVersionExpiration. Ancestor: NoncurrentVersionExpiration	Yes
NoncurrentVersionExpiration	Specifies when non-current object versions expire. Upon expiration, S3 permanently deletes the non-current object versions. You set this lifecycle configuration action on a bucket that has versioning enabled (or suspended) to request that S3 delete non-current object versions at a specific period in the object's lifetime. Type: Container Children: NoncurrentDays Ancestor: Rule	Yes, if no other action is present in the Rule.
Prefix	Object key prefix identifying one or more objects to which the rule applies. Empty prefix (<Prefix></Prefix>) indicates there is no filter based on key prefix. Note: Supports <Prefix> with and without <Filter>. (Deprecated) "Prefix": "", # No Prefix "Prefix": "documents/", PUT Bucket lifecycle with <Filter> "Filter": { "Prefix": ""}"Filter": { "Prefix": "documents/"} There can be at most one Prefix in a lifecycle rule Filter. Type: String Ancestor: Filter or And (if you specify multiple filters such as a prefix and one or more tags)	No
Rule	Container for a lifecycle rule. A lifecycle configuration can contain as many as 1,000 rules. Type: Container Ancestor: LifecycleConfiguration	Yes
Status	if Enabled, S3 executes the rule as scheduled. If Disabled, S3 ignores the rule. Type: String Ancestor: Rule Valid values: Enabled, Disabled.	Yes
Value	Specifies the value for a tag key. Each object tag is a key-value pair. Tag value can be up to 256 Unicode characters in length. Type: String Ancestor: Tag	Yes, if <Tag> parent is specified.