4259 links
  • Arnaud's links
  • Home
  • RSS Feed
  • Tag cloud
  • Login
search: tag:aws — newest first
  • GitHub - jkehler/awslambda-psycopg2
    September 2, 2019 - permalink - https://github.com/jkehler/awslambda-psycopg2
    aws lambda postgresql psycopg2 python
  • GitHub - claranet/terraform-aws-lambda: Terraform module for AWS Lambda functions
    September 2, 2019 - permalink - https://github.com/claranet/terraform-aws-lambda
    aws lambda terraform
  • Working with GPUs on Amazon ECS - Amazon Elastic Container Service
    August 21, 2019 - permalink - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-gpu.html
    aws gpu
  • API failures Error Messages - Amazon Elastic Container Service

    Your task definition contains a parameter that requires a specific container instance attribute that is not available on your container instances. For example, if your task uses the awsvpc network mode, but there are no instances in your specified subnets with the ecs.capability.task-eni attribute. For more information about which attributes are required for specific task definition parameters and agent configuration variables, see Task Definition Parameters and Amazon ECS Container Agent Configuration.

    August 21, 2019 - permalink - https://docs.amazonaws.cn/en_us/AmazonECS/latest/developerguide/api_failures_messages.html
    aws ecs
  • Jerry Hargrove | History of Amazon Web Services

    SQS first service :|

    August 15, 2019 - permalink - https://www.awsgeek.com/pages/AWS-History/
    aws history
  • Note: ssm links

    https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html

    https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-sessions-start.html#sessions-start-ssh

    https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html

    August 10, 2019 - permalink - https://links.infomee.fr/shaare/Hgv05w
    aws ssm
  • Managing PostgreSQL users and roles | AWS Database Blog

    A faire en superuser:

    -- Revoke privileges from 'public' role
    REVOKE CREATE ON SCHEMA public FROM PUBLIC;
    REVOKE ALL ON DATABASE mydatabase FROM PUBLIC;

    -- Create schema
    CREATE SCHEMA myschema

    -- Read-only role
    CREATE ROLE readonly;
    GRANT CONNECT ON DATABASE mydatabase TO readonly;
    GRANT USAGE ON SCHEMA myschema TO readonly;
    GRANT SELECT ON ALL TABLES IN SCHEMA myschema TO readonly;
    ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT SELECT ON TABLES TO readonly;

    -- Read/write role
    CREATE ROLE readwrite;
    GRANT CONNECT ON DATABASE mydatabase TO readwrite;
    GRANT USAGE, CREATE ON SCHEMA myschema TO readwrite;
    GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA myschema TO readwrite;
    ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO readwrite;
    GRANT USAGE ON ALL SEQUENCES IN SCHEMA myschema TO readwrite;
    ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT USAGE ON SEQUENCES TO readwrite;

    -- Users creation
    CREATE USER reporting_user1 WITH PASSWORD 'some_secret_passwd';
    CREATE USER reporting_user2 WITH PASSWORD 'some_secret_passwd';
    CREATE USER app_user1 WITH PASSWORD 'some_secret_passwd';
    CREATE USER app_user2 WITH PASSWORD 'some_secret_passwd';

    -- Grant privileges to users
    GRANT readonly TO reporting_user1;
    GRANT readonly TO reporting_user2;
    GRANT readwrite TO app_user1;
    GRANT readwrite TO app_user2;

    August 3, 2019 - permalink - https://aws.amazon.com/fr/blogs/database/managing-postgresql-users-and-roles/
    aws postgresql rds
  • AWS Fargate Pricing Calculator

    0,25 vCPU + 0,5GB = 9,010$ (1 month)

    July 23, 2019 - permalink - http://fargate-pricing-calculator.site.s3-website-us-east-1.amazonaws.com/
    aws fargate
  • Terraform: terraform_remote_state - Terraform by HashiCorp
    July 12, 2019 - permalink - https://www.terraform.io/docs/providers/terraform/d/remote_state.html
    aws data terraform
  • Announcing Amazon VPC Traffic Mirroring for Amazon EC2 Instances
    July 3, 2019 - permalink - https://aws.amazon.com/about-aws/whats-new/2019/06/announcing-amazon-vpc-traffic-mirroring-for-amazon-ec2-instances/
    aws mirror network
  • AWS Systems Manager Session Manager - AWS Systems Manager

    interessant

    June 28, 2019 - permalink - https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html
    aws ssh ssm
  • Secure your instances with multi-factor authentication | AWS Startups Blog
    June 12, 2019 - permalink - https://aws.amazon.com/fr/blogs/startups/securing-ssh-to-amazon-ec2-linux-hosts/
    aws mfa ssh
  • Tutorial: Delegate Access Across AWS Accounts Using IAM Roles - AWS Identity and Access Management

    Gestion multi compte aws

    May 30, 2019 - permalink - https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html
    account aws
  • GitHub - mlabouardy/komiser: Cloud Environment Inspector đź‘®
    May 20, 2019 - permalink - https://github.com/mlabouardy/komiser
    aws cost
  • cni-ipvlan-vpc-k8s/README.md at master · lyft/cni-ipvlan-vpc-k8s · GitHub
    May 14, 2019 - permalink - https://github.com/lyft/cni-ipvlan-vpc-k8s/blob/master/README.md
    aws cni k8s
  • AWS re:Invent 2014 | (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) - YouTube
    May 12, 2019 - permalink - https://www.youtube.com/watch?v=Cb2czfCV4Dg
    aws boto3 python
  • Amazon S3 Path Deprecation Plan – The Rest of the Story | AWS News Blog
    May 9, 2019 - permalink - https://aws.amazon.com/fr/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
    aws s3
  • Note: EKS max pods by EC2

    Le nombre de pods par worker node dépend du type d'EC2 utilisé qui détermine combien d'interfaces secondaires on a à notre disposition et de combien d'ips sur chaque interfaces on peut allouer.

    Exemple avec une t2.small, on a 2 interfaces secondaires et sur chacune on peut allouer 4 ips. On peut donc lancer 8 pods maximum sur une t2.small

    Il faut aussi savoir qu'un cluster EKS va faire tourner de base quelques pods :
    - un deployment pour coredns avec un replicaset=2 par défaut
    - un daemonset spécifique à aws (awsnode)
    - un daemonset pour le kubeproxy

    Ce qui équivaut à 2 pods occupés sur chaque worker node (à cause des deux daemonset) et 2 pod supplémentaire lancé sur la totalité du cluster (pour coredns).

    Il faut aussi compter un pod pour le dashboard, un pod pour le metrics-server et surement 2 pods pour l'external-dns, sans compter les ingress

    Tout ça pour dire que ce n'est pas facile d'avoir un "petit" cluster EKS : on va vite être limité par le nombre d'ip qui est assez faible sur les EC2 les plus modestes et on va vite être obligé de lancer des EC2 supplémentaire juste pour avoir des ip

    February 24, 2019 - permalink - https://links.infomee.fr/shaare/j-DVoQ
    aws eks k8s
  • Secrets Management within AWS ECS – Hacker Noon

    olded by https://docs.aws.amazon.com/fr_fr/AmazonECS/latest/developerguide/specifying-sensitive-data.html

    January 17, 2019 - permalink - https://hackernoon.com/secrets-management-within-aws-ecs-1b6975819ccd
    aws docker ecs secret ssm
  • What is Amazon Elastic Container Service? - Amazon Elastic Container Service
    November 15, 2018 - permalink - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
    aws docker ecs
  • What Is Amazon VPC? - Amazon Virtual Private Cloud
    November 15, 2018 - permalink - https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html
    aws network vpc
  • awslabs/amazon-ecr-credential-helper: Automatically gets credentials for Amazon ECR on docker push/docker pull
    September 20, 2018 - permalink - https://github.com/awslabs/amazon-ecr-credential-helper
    aws ecr
  • AWS vs Azure vs Google vs IBM vs Oracle vs Alibaba | A detailed comparison and mapping between various cloud services
    August 10, 2018 - permalink - http://comparecloud.in/
    aws cloud compare
  • Note: Containers on aws

    beanstalk : https://aws.amazon.com/fr/elasticbeanstalk/

    EKS : https://aws.amazon.com/eks/

    Fargate : https://aws.amazon.com/fargate/

    July 2, 2018 - permalink - https://links.infomee.fr/shaare/78xRRg
    amazon aws cicd container docker
  • Netflix: What Happens When You Press Play?

    from ifttt pocket

    December 14, 2017 - permalink - http://highscalability.com/blog/2017/12/11/netflix-what-happens-when-you-press-play.html
    aws netflix
  • sysadvent: Day 9 - Using Kubernetes for multi-provider, multi-region batch jobs
    December 9, 2017 - permalink - http://sysadvent.blogspot.fr/2017/12/day-9-using-kubernetes-for-multi.html
    aws docker k8s
  • Installing Kubernetes on AWS with kops | Kubernetes
    • https://www.youtube.com/watch?v=52G2T1V_UJ4
    November 23, 2017 - permalink - https://kubernetes.io/docs/getting-started-guides/kops/
    aws k8s
  • Note: get aws meta

    curl -s http://169.254.169.254/latest/meta-data/

    or

    /opt/aws/bin/ec2-metadata

    November 22, 2017 - permalink - https://links.infomee.fr/shaare/hpMq3w
    aws meta
  • Using Temporary Security Credentials to Request Access to AWS Resources - AWS Identity and Access Management

    Use role in script

    November 21, 2017 - permalink - http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html?shortFooter=true#using-temp-creds-sdk-cli
    aws bash cli role
  • Amazon ECS prĂ©sente le mode de mise en rĂ©seau des conteneurs AWSVPC pour une prise en charge des fonctionnalitĂ©s de mise en rĂ©seau complètes
    November 17, 2017 - permalink - https://aws.amazon.com/fr/about-aws/whats-new/2017/11/amazon-ecs-introduces-awsvpc-networking-mode-for-containers-to-support-full-networking-capabilities/
    aws ecs
  • Note:

    SYNOPSIS
    get-login
    [--registry-ids [...]]
    [--include-email | --no-include-email]

    OPTIONS
    --registry-ids (string) A list of AWS account IDs that correspond to
    the Amazon ECR registries that you want to log in to.

       --include-email | --no-include-email (boolean) Specify if the '-e' flag
       should  be  included in the 'docker login' command. The '-e' option has
       been deprecated and is removed in docker version 17.06 and  later.  You
       must specify --no-include-email if you're using docker version 17.06 or
       later. The default behavior is to include the '-e' flag in the  'docker
       login' output.
    
    October 6, 2017 - permalink - https://links.infomee.fr/shaare/ECje8g
    aws ecr
  • Note: List users and their inline attached policies

    for user in $(aws iam list-users|jq '.Users|.[]|.UserName' -r); do echo $user;aws iam list-user-policies --user-name $user; done

    September 22, 2017 - permalink - https://links.infomee.fr/shaare/xhJZ2g
    aws iam
  • How Do I Create a Lifecycle Policy for an S3 Bucket? - Amazon Simple Storage Service

    To apply this lifecycle rule to all objects in the bucket, choose Next.

    That's why wildcard was not working :D

    September 22, 2017 - permalink - http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html
    aws policy s3
  • GitHub - abutaha/aws-es-proxy: aws-es-proxy is a small web server application sitting between your HTTP client (browser, curl, etc...) and Amazon Elasticsearch service.

    Amazon Elasticsearch access control may be based on IAM account with signed request mechanism
    One way not to rewrite all applications is using such a proxy

    September 5, 2017 - permalink - https://github.com/abutaha/aws-es-proxy
    aws elasticsearch es
  • Note: List item in dynamo db table

    aws dynamodb scan --table-name foo

    August 18, 2017 - permalink - https://links.infomee.fr/shaare/AqSZDA
    aws dynamodb
  • Note: Delete dynamo db item in cmdline

    aws dynamodb delete-item --table-name foo --key "{\"id\":{\"S\":\"$id\"}}"

    August 18, 2017 - permalink - https://links.infomee.fr/shaare/bx3KXg
    aws dynamodb
  • upload-server-certificate — AWS CLI 1.11.124 Command Reference

    --path

    July 25, 2017 - permalink - http://docs.aws.amazon.com/cli/latest/reference/iam/upload-server-certificate.html
    aws iam ssl
  • SQS — Boto 3 Docs 1.4.4 documentation

    On peut "remettre" ou plutot rendre disponible de nouveau un message SQS en changeant sa visibility timeout Ă  0

    July 12, 2017 - permalink - http://boto3.readthedocs.io/en/latest/reference/services/sqs.html#SQS.Client.change_message_visibility
    aws sqs
  • Note: Count object in bucket/folder

    aws s3 ls s3://bucket/path/ --recursive --summarize | grep "Total Objects:"

    July 12, 2017 - permalink - https://links.infomee.fr/shaare/rcG0hg
    aws count s3
  • set-repository-policy — AWS CLI 1.11.117 Command Reference

    Truc con : on ne peut pas utiliser le résultat du get-repository-policy dans le set-repository-policy pour cloner.

    Il faut au passage enlever les \n qui trainent dans la réponse

    July 11, 2017 - permalink - http://docs.aws.amazon.com/cli/latest/reference/ecr/set-repository-policy.html
    aws ecs
  • Amazon Glacier Direct Upload vs Glacier Upload via Amazon S3

    So you can put your data into glacier with 2 differents ways:
    1) directly to glacier via API
    2) Store them to s3 then with a management policy, it'll go to Glacier

    Warning : Huge cost when you download from glacier and when you delete before 3 months

    June 27, 2017 - permalink - https://www.cloudberrylab.com/blog/compare-amazon-glacier-direct-upload-and-glacier-upload-through-amazon-s3/
    aws glacier s3
  • Creating a Redis (cluster mode enabled) Cluster with Replicas from Scratch - Amazon ElastiCache
    June 20, 2017 - permalink - http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Replication.CreatingReplGroup.NoExistingCluster.Cluster.html
    aws redis
  • Creating a Redis (cluster mode disabled) Cluster with Replicas from Scratch - Amazon ElastiCache
    June 20, 2017 - permalink - http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Replication.CreatingReplGroup.NoExistingCluster.Classic.html
    aws redis
  • Replication: Redis (cluster mode disabled) vs. Redis (cluster mode enabled) - Amazon ElastiCache
    June 20, 2017 - permalink - http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Replication.Redis-RedisCluster.html
    aws redis
  • Replication: Multi-AZ with Automatic Failover (Redis) - Amazon ElastiCache
    June 19, 2017 - permalink - http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoFailover.html
    aws redis
  • Note: some draft about monitoring beanstalk applications health with boto3 python script

    import pprint
    p = pprint.PrettyPrinter(indent=4)
    p.pprint(x)

    or

    import pprint
    pprint.pformat(x)

    import logging
    import pprint
    logging.info(pprint.pformat(dict))

    $ cat monitor_beanstalk.py

    !/bin/python

    import boto3
    import pprint
    pp = pprint.PrettyPrinter(indent=4)

    List all env and status and instances health

    client = boto3.client('elasticbeanstalk')

    envs = client.describe_environments()['Environments']

    pp.pprint(envs)

    for env in envs:
    print 'ApplicationName: {} EnvironmentName: {} Health: {} HealthStatus: {} Status: {}'.format(env['ApplicationName'].ljust(30),env['EnvironmentName'].ljust(30),env['Health'].ljust(10),env.get('HealthStatus', 'N/A').ljust(10),env['Status'].ljust(10))
    if (env['Health'] != 'Green') or (env.get('HealthStatus', 'N/A') != 'Ok' and env.get('HealthStatus', 'N/A') != 'N/A'):
    print '\nProblem'
    details = client.describe_environment_health(EnvironmentName=env['EnvironmentName'],AttributeNames=['All'])
    #pp.pprint(details)
    print details['Causes']
    print details['InstancesHealth']
    print '\n'

    June 15, 2017 - permalink - https://links.infomee.fr/shaare/eFFzMg
    aws beanstalk boto nomarkdown pprint python
  • Note: EFS size by api

    aws efs describe-file-systems| jq '.FileSystems|.[]|[.Name, .SizeInBytes.Timestamp, .SizeInBytes.Value]' -c

    Retourne une ligne par EFS
    Sur chaque ligne, un array avec :
    [0] = nom de l'efs
    [1] = timestamp du moment où la taille a été calculée
    [2] = la taille en bytes

    Pour avoir la taille en GB :

    aws efs describe-file-systems| jq '.FileSystems|.[]|[.Name, .SizeInBytes.Timestamp, .SizeInBytes.Value / 1024 /1024 / 1024]' -c

    aws efs describe-file-systems| jq '.FileSystems|.[]|[.Name, .SizeInBytes.Value / 1024 /1024 / 1024]' -c

    June 13, 2017 - permalink - https://links.infomee.fr/shaare/MzuW2A
    aws efs size
  • Example 2: Bucket Owner Granting Cross-Account Bucket Permissions - Amazon Simple Storage Service

    Donc pour autoriser un compte externe, on va créer une bucket policy sur notre bucket pour autoriser "arn:aws:iam::account_id:root" ou plus précis sur l'user arn:aws:iam::account_id:user/foobar ou le role

    C'est le compte en face qui va décider qui a le droit de venir sur notre bucket avec des user policy standard (quand on est dans le contexte du compte en face, c'est comme si le bucket nous appartenait)

    Exemple bucket policy Ă  mettre sur le BUCKET de l'account A pour autoriser l'account xxx en RW

    {
    "Version": "2012-10-17",
    "Statement": [

        {
            "Sid": "Allow account_xx on aws account xxx RW",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
            },
            "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
            "Resource": "arn:aws:s3:::BUCKET"
        },
        {
            "Sid": "Allow account_xx on aws account xxx RW",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
            },
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::BUCKET/*"
        }
    
    
    
    
    ]
    

    }

    Pour Read only, remplacer action du deuxieme bloc par "Action": ["s3:Get","s3:List"],

    June 12, 2017 - permalink - http://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example2.html
    aws cross iam nomarkdown policy s3
  • Copy between S3 buckets w/ different accounts · GitHub
    June 8, 2017 - permalink - https://gist.github.com/ushu/7217693
    aws s3
  • The Challenges of EFS

    one more

    June 8, 2017 - permalink - https://convox.com/blog/challenges-of-efs/
    aws efs
â–Ľ Load 50 more