I needed to push files from a few ec2 instances into an s3 bucket. These weren't my systems, and I won't be around to support them in a couple of months anyway, so I wanted to do it with a simple, stand-alone script that didn't require me to install anything else on them.
You need to assign the IAM role when you first launch an instance. Fortunately the instances were set up with an IAM role already, so I just needed to make sure they had the right permissions.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::EXAMPLE-BUCKET/*"
}
]
}
Normal beginning:
#!/usr/bin/perl -w
use strict;
Get file and bucket names from command line:
unless (@ARGV == 3) { die "Usage: put-into-s3 BUCKET LOCALFILE REMOTEFILE\n"; }
my($bucket, $localfile, $remotefile) = @ARGV;
unless (-f $localfile) { die "$localfile isn't an existing file\n"; }
$remotefile =~ s#^([^/])#/$1#;
my $remotehost = "$bucket.s3.amazonaws.com";
my $url = "https://$remotehost$remotefile";
my $resource = "/$bucket$remotefile";
RFC 2822 date for the HTTP header.
chomp(my $date = `/bin/date -R`);
Here's the first big bit, getting credentials from the IAM role. The first curl gets the name of the IAM role on that server, and then the second one gets the credential information. Then I pull out the pieces I want.
sub get_creds(@) {
my $iaminfo = "http://169.254.169.254/latest/meta-data/iam/security-credentials/";
chomp(my $role = `/usr/bin/curl -s $iaminfo`);
my $iamurl = "$iaminfo$role";
my $creds = `/usr/bin/curl -s $iamurl`;
my @results = ();
for my $key (@_) {
if ($creds =~ /"$key" : "([^"]+)"/) {
push(@results,$1);
} else {
die "no key $key found in $creds\n";
}
}
return @results;
}
my($accesskeyid, $secret, $token) = get_creds('AccessKeyId','SecretAccessKey','Token');
The second big bit, hashing the signature.
open(SSL,
"/bin/echo -n 'PUT\n\n\n$date\nx-amz-security-token:$token\n$resource' |" .
"/usr/bin/openssl sha1 -binary -hmac $secret | /usr/bin/base64 |"
);
chomp(my $signature = <SSL>);
close SSL;
And finally, sending the upload to S3.
system(
'/usr/bin/curl',
'-H', "Date: ${date}",
'-H', "Authorization: AWS ${accesskeyid}:${signature}",
'-L',
'-H', "content-type: ",
'-T', $localfile,
'-H', "x-amz-security-token: $token",
$url
);
I couldn't quickly google a complete solution, but these two links helped immensely.