IT/AWS/S3
From Lewis Consultancy Wiki
Revision as of 09:01, 1 September 2021 by Clewis (talk | contribs) (→Tag files ( set tag archive=yes ))
Contents
Tag objects in an S3 bucket based on size
It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier based on tag.
Get full filelist of bucket and store
aws s3 ls s3://bucketname/path --recursive > /tmp/fullfilelist.txt
Select only files greater than x bytes
cat /tmp/fullfilelist.txt | awk '{ if ($3> 5000000) print $4}' > /tmp/largefilenames.txt
Tag files ( set tag archive=yes )
cat /tmp/largefilenames.txt | while read fn; do aws s3api put-object-tagging --bucket bucketname --key "$fn" --tagging '{"TagSet": [{ "Key": "archive", "Value": "yes" }]}'; done