Sunday, 22 October 2017

aws

1. aws cli credential (also see item 9)
  • '~/.aws/config' file, or
    '~/.aws/credentials' file, or
    'AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY' env variable, or
    ec2 instance profile credentials
  • 'credentials' stores access keys
  • 'config' stores other configuration like region and output format
  • no conflict between 'credentials' and 'config', aws cli or java sdk will look at both file
  • when using aws cli, set 'AWS_PROFILE' env variable or use --profile flag in cli command if profile is not default
  • when using java sdk, use ProfileCredentialsProvider if profile is not default

2. iam user/group/role/policy/profile
  • user is used to sign in  (aws iam list-users)
  • group is collection of users
  • role is collection of policies (user can have multiple roles)
  • policy=permission
  • profile is a settings file

3. arn 
  • amazon resource name
  • arn:partition:service:region:account:resource        //format
  • arn:aws:iam::998355367879:user/test-user            //example: user

4. kinesis
  • streaming data service
  • consumer: firehose, data analytics etc
  • pip3 install kines
  • kines walk streamotion-gam-datalake-nonprod  000000000120 -l -f    //000000000120 is one of the shards id. shard == partition 

5. commands
  • aws configure list-profiles         //show all profiles
  • aws configure list                      //show current profile
  • aws sts get-caller-identity         //current user, role

6. profile
  • [default] is used when running cmd without --profile param
  • export AWS_PROFILE=user1                  //use the named profile

7. cli settings precedence (which override which)
  • command line option                    //--profile
  • environment variable                    //export 
  • credentials file
  • config file

8. ec2 key pair
  • when ec2 instance boost, public key is stored in instance
  • when connecting to ec2 instance, you must specify private key

9. set credentials when use java sdk or aws cli
  • option 1: use '~/.aws/credentials' file to set up credential (not 'config' file)
[default]
role_arn = arn:aws:iam::111111111111:role/group-xxxxx-xxxxx
source_profile = okta
region = ap-southeast-2

[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY

  • option 2: use aws environment variable to set up credentials
export AWS_ACCESS_KEY_ID=your_access_key_id
export AWS_SECRET_ACCESS_KEY=your_secret_access_key
export AWS_REGION=your_aws_region
  • aws env variable take precedence over ~/.aws/credentials, but in general, it's best practice to store credentials in the ~/.aws/credentials file, and load the credentials in your code using the AWSCredentialsProvider class
AWSCredentialsProvider credentialsProvider = new ProfileCredentialsProvider();


10. aws lambda vs ec2
  • ec2: need to provision container, orchestration, scaling
  • lambda: taken care of by aws

  • ec2: need to take care of security layer access
  • lambda: taken care of by aws

  • ec2: combined with ALB, has no timeout limit
  • lambda: combined with api gateway, has timeout limit of 15min for lambda and 30 sec for gateway

  • ec2: need to config auto scaling group
  • lambda: taken care of by aws

  • ec2: always available
  • lambda: on demand

  • ec2: cold start not needed unless for a new container
  • lambda: cold start needed

11. api gateway
  • default max integration timeout limit 30 sec

12. skill used at work
  • management console, cli, sdk
  • s3, secret manager, kinesis, sqs, nosql
  • ec2 - elastic compute cloud (vm)
  • ecs - elastic container service (docker)

reference

No comments:

Post a Comment