IT/AWS/S3

Tag objects in an S3 bucket based on size

It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier based on tag.

Get full filelist of bucket and store

aws s3 ls s3://bucketname/path --recursive > /tmp/fullfilelist.txt

Select only files greater than x bytes

cat /tmp/fullfilelist.txt | awk '{ if ($3> 5000000)  print $4}' > /tmp/largefilenames.txt

Tag files (set tag archive=yes

cat /tmp/largefilenames.txt | while read fn; do aws s3api put-object-tagging --bucket inview-nas1 --key "$fn" --tagging '{"TagSet": [{ "Key": "archive", "Value": "yes" }]}'; done