Lifecycle Policies

Lifecycle Policies are a way to automatically delete objects from your servinga S3 Object Storage buckets. This is especially useful if you know in advance that you will no longer need uploaded files after a certain amount of time like e.g. with backups or log files.

In order to use and enable a Lifecycle Policy in your bucket, you first need to create a XML policy file that tells the servinga S3 Object Storage which files to delete and when.

info

Lifecycle Policies are being automatically executed throughout the day. The feature doesn't provide the guarantee that an object is deleted immediately right after its expiration. Instead, the deletion process can take up to 24 hours, depending on general load.

Policy Structure

A Lifecycle Policy is a XML file that contains rules for deleting objects in the bucket the policy is applied to. You can only apply a single lifecycle policy to a bucket, but one policy can contain up to 1,000 rules.

The structure of the XML file is as follows:

<LifecycleConfiguration>
    <Rule>
         <Element />
    </Rule>
    <Rule>
         <Element />
         <Element />
    </Rule>
</LifecycleConfiguration>

Please refer to Amazon's documentation for more information on how to configure the lifecycle of objects within a bucket.

Example

A common example and use-case is the automatic deletion of backups after a certain amount of time. That way you can simply put the backup files into a bucket and don't have to worry about deleting them manually and accumulating more and more storage space.

Given the assumption that we have a "directory" in our bucket called backups and that we want to delete all files in that directory after 30 days, the following policy could be applied:

<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Rule>
        <ID>70ab059a-9d79-4188-8d0c-a28363dfddc7</ID>
        <Status>Enabled</Status>
        <Expiration>
            <Days>30</Days>
        </Expiration>
        <Filter>
            <Prefix>backups/</Prefix>
        </Filter>
    </Rule>
</LifecycleConfiguration>

The rule's ID can be freely chosen, but it must be unique within the bucket. For the simplicity's sake, we have chosen a random UUID here.

In order to apply the lifecycle policy, you can use a third party tool like s3cmd which offers a command line interface sub-command to apply a lifecycle policy to a bucket.

s3cmd setlifecycle policy.xml s3://your-bucket