Difference in boto3 between resource, client, and session?

by

Client and Resource are two different abstractions within the boto3 SDK for making AWS service requests. If you want to make API calls to an AWS service with boto3, then you do so via a Client or a Resource.

You would typically choose to use either the Client abstraction or the Resource abstraction, but an application can use both, as needed. I’ve outlined the differences between Client and Resource below to help readers decide which to use.

Session is largely orthogonal to the concepts of Client and Resource (but is used by both).

Here’s some more detailed information on what Client, Resource, and Session are all about.

Client:

  • this is the original boto3 API abstraction
  • it provides low-level AWS service access
  • all AWS service operations are supported by clients
  • it exposes botocore client to the developer
  • it typically maps 1:1 with the AWS service API
  • it exposes snake-cased method names (e.g. ListBuckets API => list_buckets method)
  • it is generated from an AWS service description

Here’s an example of client-level access to an S3 bucket’s objects:

import boto3

client = boto3.client('s3')
response = client.list_objects_v2(Bucket="mybucket")
for content in response['Contents']:
    obj_dict = client.get_object(Bucket="mybucket", Key=content['Key'])
    print(content['Key'], obj_dict['LastModified'])

Note: this client-level code is limited to listing at most 1000 objects. You would have to use a paginator, or implement your own loop, calling list_objects_v2() repeatedly with a continuation marker if there were more than 1000 objects.

OK, so that’s the low-level Client interface. Now onto the higher-level (more abstract) Resource interface.

Resource:

  • this is the newer boto3 API abstraction
  • it provides high-level, object-oriented API
  • it does not provide 100% API coverage of AWS services
  • it uses identifiers and attributes
  • it has actions (operations on resources)
  • it exposes sub-resources and collections of AWS resources
  • it is generated from an AWS resource description

Here’s the equivalent example using resource-level access to an S3 bucket’s objects (all):

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('mybucket')
for obj in bucket.objects.all():
    print(obj.key, obj.last_modified)

Note: in this case you do not have to make a second API call to get the objects; they’re available to you as a collection on the bucket. These collections of sub-resources are lazily-loaded.

You can see that the Resource version of the code is much simpler, more compact, and has more capability (for example it does pagination for you and it exposes properties instead of a raw dictionary). The Client version of the code would actually be more complicated than shown above if you wanted to include pagination.

Finally, onto Session which is fundamental to both Client and Resource and how both get access to AWS credentials, for example.

Session:

  • stores configuration information (primarily credentials and selected region)
  • allows you to create service clients and resources
  • boto3 creates a default session for you when needed

A useful resource to learn more about these boto3 concepts is the introductory re:Invent video.

More Related Contents:

  1. How to write a file or data to an S3 object using boto3
  2. Boto3 to download all files from a S3 Bucket
  3. How to import a text file on AWS S3 into pandas without writing to disk
  4. Complete scan of dynamoDb with boto3
  5. How to handle errors with boto3?
  6. Save Dataframe to csv directly to s3 Python
  7. How to choose an AWS profile when using boto3 to connect to CloudFront
  8. How to save S3 object to a file using boto3
  9. Read file content from S3 bucket with boto3
  10. check if a key exists in a bucket in s3 using boto3
  11. Error “Read-only file system” in AWS Lambda when downloading a file from S3
  12. How to SSH and run commands in EC2 using boto3?
  13. Listing contents of a bucket with boto3
  14. Retrieving subfolders names in S3 bucket from boto3
  15. How to read a list of parquet files from S3 as a pandas dataframe using pyarrow?
  16. How to upload File in FastAPI, then to Amazon S3 and finally process it?
  17. Can I use boto3 anonymously?
  18. Getting S3 objects’ last modified datetimes with boto
  19. How to specify credentials when connecting to boto3 S3?
  20. Mocking boto3 S3 client method Python
  21. Open S3 object as a string with Boto3
  22. how to copy s3 object from one bucket to another using python boto3
  23. How to return a PDF file from in-memory buffer using FastAPI?
  24. Reading an JSON file from S3 using Python boto3
  25. Download a folder from S3 using Boto3
  26. In python how can we append run time reults to a list?
  27. Most efficient way to build single-element list from string with or without spaces? [closed]
  28. Groupby value counts on the dataframe pandas
  29. How to display a float with two decimal places?
  30. Why is dictionary ordering non-deterministic?

Leave a Comment