4259 links
  • Arnaud's links
  • Home
  • RSS Feed
  • Tag cloud
  • Login
search: tag:nomarkdown — newest first
  • Auto activate and deactivate python venv using zsh - DEV Community

    python_venv() {
    MYVENV=./venv
    # when you cd into a folder that contains $MYVENV
    [[ -d $MYVENV ]] && source $MYVENV/bin/activate > /dev/null 2>&1
    # when you cd into a folder that doesn't
    [[ ! -d $MYVENV ]] && deactivate > /dev/null 2>&1
    }
    autoload -U add-zsh-hook
    add-zsh-hook chpwd python_venv

    May 20, 2022 - permalink - https://dev.to/moniquelive/auto-activate-and-deactivate-python-venv-using-zsh-4dlm
    nomarkdown python venv zsh
  • Encrypting and decrypting archives with 7-Zip | Enable Sysadmin

    $ tar -cf directory.tar
    $ 7z a -p -mhe=on directory.tar.7z directory.tar
    ======================
    7z x archive.tar.7z
    tar xvf archive.tar

    March 8, 2022 - permalink - https://www.redhat.com/sysadmin/encrypting-decrypting-7zip
    7z nomarkdown
  • BInary search algo

    def binary_search(l, item):
    first = 0
    last = len(l)-1
    found = False

    while first<=last and not found:
        midpoint = round((first + last)/2)
        if l[midpoint] == item:
            found = True
        else:
            if item < l[midpoint]:
                last = midpoint-1
            else:
                first = midpoint+1
    
    return found
    

    input = [0, 1, 2, 8, 13, 17, 19, 32, 42,]

    print(binary_search(input, 3)) # found: False
    print(binary_search(input, 13)) # fount: True

    February 6, 2022 - permalink - https://links.infomee.fr/shaare/GnzD5Q
    algo nomarkdown search
  • Quick sort algo

    from random import randrange

    input = [10, 5, 2, 3, 7, 0, 9, 12]

    def quicksort(arr):
    if len(arr) < 2:
    return arr
    else:
    rand = randrange(0, len(arr)) # grab a random index
    pivot = arr.pop(rand)
    less = [i for i in arr if i <= pivot]
    greater = [i for i in arr if i > pivot]
    return quicksort(less) + [pivot] + quicksort(greater)

    print("sorted: ", quicksort(input))

    February 6, 2022 - permalink - https://www.integralist.co.uk/posts/algorithmic-complexity-in-python/
    algo nomarkdown recursiv sort
  • python - How to search and replace text in a file? - Stack Overflow

    Sometime you cant use sed :(

    Read in the file

    with open('file.txt', 'r') as file :
    filedata = file.read()

    Replace the target string

    filedata = filedata.replace('ram', 'abcd')

    Write the file out again

    with open('file.txt', 'w') as file:
    file.write(filedata)

    July 7, 2021 - permalink - https://stackoverflow.com/questions/17140886/how-to-search-and-replace-text-in-a-file
    nomarkdown python replace
  • Configuring Docker to not use the 172.17.0.0 range - Server Fault

    Pour changer les subnets utilisés par les network bridge users

    nano /etc/docker/daemon.json

    Add lines:

    {
    "default-address-pools":
    [
    {"base":"10.10.0.0/16","size":24}
    ]
    }

    April 12, 2021 - permalink - https://serverfault.com/questions/916941/configuring-docker-to-not-use-the-172-17-0-0-range
    docker nomarkdown
  • Note: Replace literal \n with actual newline

    sed 's/\n/\n/g' /tmp/t

    September 30, 2020 - permalink - https://links.infomee.fr/shaare/cXcMlg
    backslash literal log logs n nomarkdown replace sed
  • Force deployment rolling-update · Issue #27081 · kubernetes/kubernetes

    kubectl patch deployment web -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"date +'%s'\"}}}}}"

    mieux :
    kubectl rollout restart daemonset/filebeat-filebeat
    kubectl rollout restart deployment/web

    March 25, 2019 - permalink - https://github.com/kubernetes/kubernetes/issues/27081
    k8s nomarkdown patch rolling update
  • Configure HAProxy for userlists

    Create SHA512 passwords

    make sure to use a leading space so that the command is not stored in your bash history!!

    mkpasswd -m sha-512 password1

    generates -> $6$yMgsow58.g/Z$mBjHfdVzqcF/LN.iwV23Eyqg.yGPTsp9pOwaStsJ6c4I4zL7BhucVVAkv5guf7OVRr8Pw0mHF4NrWBRCG5ci7/

    mkpasswd -m sha-512 password2

    generates -> $6$RZ86vRkQ$aRKN1HOsk6bDHBbMhS7jSo/p1NGFl4PvwY3KpU.72i./LvITi41nL84EkxOFXl.6Bmhynj/L7pYbfF0rUHtOB0

    Edit /etc/haproxy/haproxy.cfg

    userlist UsersFor_Ops
    group AdminGroup users userone,usertwo
    user userone password $6$yMgsow58.g/Z$mBjHfdVzqcF/LN.iwV23Eyqg.yGPTsp9pOwaStsJ6c4I4zL7BhucVVAkv5guf7OVRr8Pw0mHF4NrWBRCG5ci7/
    user usertwo password $6$RZ86vRkQ$aRKN1HOsk6bDHBbMhS7jSo/p1NGFl4PvwY3KpU.72i./LvITi41nL84EkxOFXl.6Bmhynj/L7pYbfF0rUHtOB0

    ...

    backend myserver
    mode http
    server myserver 1.1.1.1:80
    acl AuthOkay_Ops http_auth(UsersFor_Ops)
    http-request auth realm MyAuthRealm if !AuthOkay_Ops

    March 16, 2018 - permalink - https://gist.github.com/Iristyle/5005653
    haproxy nomarkdown
  • Note: some draft about monitoring beanstalk applications health with boto3 python script

    import pprint
    p = pprint.PrettyPrinter(indent=4)
    p.pprint(x)

    or

    import pprint
    pprint.pformat(x)

    import logging
    import pprint
    logging.info(pprint.pformat(dict))

    $ cat monitor_beanstalk.py

    !/bin/python

    import boto3
    import pprint
    pp = pprint.PrettyPrinter(indent=4)

    List all env and status and instances health

    client = boto3.client('elasticbeanstalk')

    envs = client.describe_environments()['Environments']

    pp.pprint(envs)

    for env in envs:
    print 'ApplicationName: {} EnvironmentName: {} Health: {} HealthStatus: {} Status: {}'.format(env['ApplicationName'].ljust(30),env['EnvironmentName'].ljust(30),env['Health'].ljust(10),env.get('HealthStatus', 'N/A').ljust(10),env['Status'].ljust(10))
    if (env['Health'] != 'Green') or (env.get('HealthStatus', 'N/A') != 'Ok' and env.get('HealthStatus', 'N/A') != 'N/A'):
    print '\nProblem'
    details = client.describe_environment_health(EnvironmentName=env['EnvironmentName'],AttributeNames=['All'])
    #pp.pprint(details)
    print details['Causes']
    print details['InstancesHealth']
    print '\n'

    June 15, 2017 - permalink - https://links.infomee.fr/shaare/eFFzMg
    aws beanstalk boto nomarkdown pprint python
  • Example 2: Bucket Owner Granting Cross-Account Bucket Permissions - Amazon Simple Storage Service

    Donc pour autoriser un compte externe, on va créer une bucket policy sur notre bucket pour autoriser "arn:aws:iam::account_id:root" ou plus précis sur l'user arn:aws:iam::account_id:user/foobar ou le role

    C'est le compte en face qui va décider qui a le droit de venir sur notre bucket avec des user policy standard (quand on est dans le contexte du compte en face, c'est comme si le bucket nous appartenait)

    Exemple bucket policy à mettre sur le BUCKET de l'account A pour autoriser l'account xxx en RW

    {
    "Version": "2012-10-17",
    "Statement": [

        {
            "Sid": "Allow account_xx on aws account xxx RW",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
            },
            "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
            "Resource": "arn:aws:s3:::BUCKET"
        },
        {
            "Sid": "Allow account_xx on aws account xxx RW",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
            },
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::BUCKET/*"
        }
    
    
    
    
    ]
    

    }

    Pour Read only, remplacer action du deuxieme bloc par "Action": ["s3:Get","s3:List"],

    June 12, 2017 - permalink - http://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example2.html
    aws cross iam nomarkdown policy s3
  • Note: s3 policy one bucket

    ReadWrite :

    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
    "Resource": [
    "arn:aws:s3:::LeBucket"
    ]
    },
    {
    "Effect": "Allow",
    "Action": [
    "s3:"
    ],
    "Resource": [
    "arn:aws:s3:::LeBucket/
    "
    ]
    }
    ]
    }

    Pour Read only, remplacer action du deuxieme bloc par "Action": ["s3:Get","s3:List"],

    May 17, 2017 - permalink - https://links.infomee.fr/shaare/Vp7r6Q
    aws iam nomarkdown s3
  • Note: mysql

    do not use utf8, its not "real utf8"

    CREATE DATABASE foo DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

    use this :

    https://mathiasbynens.be/notes/mysql-utf8mb4#utf8-to-utf8mb4

    CREATE DATABASE foo DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;

    CREATE USER 'foo'@'%' IDENTIFIED BY "password";

    GRANT ALL ON foo.* TO "foo"@"%";

    convert :

    For each database:

    ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

    For each table:

    ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

    For each column:

    ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

    (Don’t blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a VARCHAR column.)

    May 3, 2017 - permalink - https://links.infomee.fr/shaare/kmGPew
    memo mysql nomarkdown
  • Stupid Python Tricks: The KeyboardInterrupt Exception

    catching all exception in python
    import traceback

    for record in database:
    try:
    blabla
    except (KeyboardInterrupt, SystemExit):
    raise
    except Exception as e:
    # report error and proceed
    print(type(e).name)
    print(e)
    print(traceback.format_exc())

    Ne pas oublier de raise si on veut re-raise

    April 21, 2017 - permalink - http://effbot.org/zone/stupid-exceptions-keyboardinterrupt.htm
    exception nomarkdown python
  • Note: How to know which policy contains a specific action?

    Warning : this loop does a lot of call to aws api, use it with caution

    To know that I needed to list all policies and associated statements (for the default policy version)

    #!/bin/bash
    IFS=$'\n'
    for line in $(aws iam list-policies|jq '.Policies|.[]|[ .PolicyName, .Arn, .DefaultVersionId ]| @csv' -r|sed 's/","/ /g'|sed 's/"//g'); do
        name=$(echo $line|cut -d' ' -f1);
        arn=$(echo $line|cut -d' ' -f2);
        version=$(echo $line|cut -d' ' -f3);
        echo "$name"
        aws iam get-policy-version --policy-arn $arn --version-id $version
    done
    

    Put this in a script, redirect output to a file and go get grep!

    February 22, 2017 - permalink - https://links.infomee.fr/shaare/bERNcg
    aws bash for foreach iam nomarkdown policy separator
  • bash - median of column with awk - Stack Overflow

    This awk program assumes one column of numerically sorted data:

    /usr/bin/env awk

    {
    count[NR] = $1;
    }
    END {
    if (NR % 2) {
    print count[(NR + 1) / 2];
    } else {
    print (count[(NR / 2)] + count[(NR / 2) + 1]) / 2.0;
    }
    }

    Sample usage:

    sort -n data_file | awk -f median.awk

    August 2, 2016 - permalink - http://stackoverflow.com/questions/6166375/median-of-column-with-awk
    awk median nomarkdown
  • Un peu de réseau et de debug udp

    Prenons une application ouvre une socket UDP : pour chaque paquet qui arrive, elle va le traiter et passer au suivant. Si le traitement est plus cours que la cadence de reception, tout va bien, on peut travailler en "flux tendu". Pour éviter de perdre des paquets si l'application est trop lente, le kernel entretien des buffers qui ont une taille par defaut et une taille max.

    Pour consulter :

    sysctl net.core.rmem_default

    sysctl net.core.rmem_max

    sysctl net.ipv4.udp_mem

    net.core.netdev_max_backlog

    En cas de probleme, on peut augmenter la taille de ce buffer. (temporairement avec commande ou /proc/... et définitivement avec /etc/systcl.d/
    sysctl -w net.core.rmem_default=20000000
    sysctl -w net.core.rmem_max=30000000
    sysctl -w net.ipv4.udp_mem='262144 327680 393216'
    sysctl -w net.core.netdev_max_backlog=2000

    Comment savoir qu'on a des drop udp ?

    netstat -su

    Sinon il y a /proc/net/udp qui contient une ligne par socket udp avec le port (attention en hexadecimal), la taille du buffer utilisé, (s'inquiéter si rxqueue est différent de zéro) et enfin la dernière colonne, les drops !

    alors si vous avez des drops, il faut revoir l'application,.
    Augmenter la taille du buffer est utile seulement si ces drops sont du à des "burst" (des pics d'activité qui n'arrivent pas souvent), un plus grand buffer permettra d'absorber ces pics !

    En encore plus interactif : netstat -c --udp -an

    todo : trouver les commandes équivalentes avec ss

    https://github.com/etsy/statsd/issues/318
    https://www.assembla.com/spaces/LogZillaWiki/wiki/UDP_Buffers
    http://answers.splunk.com/answers/7001/udp-drops-on-linux.html

    December 10, 2014 - permalink - https://links.infomee.fr/shaare/AyHsGA
    drop kernel linux network networking nomarkdown reseau udp