If you created a presigned URL by using a temporary token, then the URL expires when the token expires, even if you created the URL with a later expiration time. For more information about how the credentials you use affect the expiration time, see Who can create a presigned URL.
So you have to use regular IAM user instead of IAM role for service generating presigned urls..? :-/
To apply this lifecycle rule to all objects in the bucket, choose Next.
That's why wildcard was not working :D
aws s3 ls s3://bucket/path/ --recursive --summarize | grep "Total Objects:"
So you can put your data into glacier with 2 differents ways:
1) directly to glacier via API
2) Store them to s3 then with a management policy, it'll go to Glacier
Warning : Huge cost when you download from glacier and when you delete before 3 months
aws s3api list-objects --bucket BUCKET_NAME --output json --query "[sum(Contents[].Size), length(Contents[])]"
Retourne la taille en bytes et le nombre d'objets
Avec un ficheir qui contient le nom des bucket :
for BUCKET_NAME in $(cat s3list); do echo -n "$BUCKET_NAME " ; region=$(aws s3api get-bucket-location --bucket $BUCKET_NAME|jq '.LocationConstraint' -r); aws s3api list-objects --region $region --bucket $BUCKET_NAME --output json --query "sum(Contents[].Size)"; done
Donc pour autoriser un compte externe, on va créer une bucket policy sur notre bucket pour autoriser "arn:aws:iam::account_id:root" ou plus précis sur l'user arn:aws:iam::account_id:user/foobar ou le role
C'est le compte en face qui va décider qui a le droit de venir sur notre bucket avec des user policy standard (quand on est dans le contexte du compte en face, c'est comme si le bucket nous appartenait)
Exemple bucket policy à mettre sur le BUCKET de l'account A pour autoriser l'account xxx en RW
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow account_xx on aws account xxx RW",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
},
"Action": ["s3:GetBucketLocation", "s3:ListBucket"],
"Resource": "arn:aws:s3:::BUCKET"
},
{
"Sid": "Allow account_xx on aws account xxx RW",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
},
"Action": [
"s3:*"
],
"Resource": "arn:aws:s3:::BUCKET/*"
}
]
}
Pour Read only, remplacer action du deuxieme bloc par "Action": ["s3:Get","s3:List"],
ReadWrite :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetBucketLocation", "s3:ListBucket"],
"Resource": [
"arn:aws:s3:::LeBucket"
]
},
{
"Effect": "Allow",
"Action": [
"s3:"
],
"Resource": [
"arn:aws:s3:::LeBucket/"
]
}
]
}
Pour Read only, remplacer action du deuxieme bloc par "Action": ["s3:Get","s3:List"],
If you enable s3 enpoint in your route table, it's kind of tricky to know if the endpoint is really working. Two things to validate:
1) traceroute tcp before and after (traceroute -T s3-us-west-1.amazonaws.com 443)
You will see more hope when endpoint not activated
2) try an s3 sync cross region with enpoint activated : it should failed since it's not supported (yet @ 2017-05-02)
I may have to use this one, but issue number is freaking me out
Don't give s3 full access policy to your app user
Prefer to allow access only for specific bucket
I wonder what's the best choice : create managed policy or simply use inline policy. I got a 1 to 1 relationship between my app-users and bucket so... inline policy looks good here
{
"Id": "S3PolicyId1",
"Statement": [
{
"Sid": "IPDeny",
"Effect": "Deny",
"Principal": {
"AWS": ""
},
"Action": "s3:",
"Resource": "arn:aws:s3:::my-wicked-awesome-bucket/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": "72.309.38.2/32"
}
}
}
]
}