Why required and optional is removed in Protocol Buffers 3

by

The usefulness of required has been at the heart of many a debate and flame war. Large camps have existed on both sides. One camp liked guaranteeing a value was present and was willing to live with its limitations but the other camp felt required dangerous or unhelpful as it can’t be safely added nor removed.

Let me explain more of the reasoning why required fields should be used sparingly. If you are already using a proto, you can’t add a required field because old application’s won’t be providing that field and applications in general don’t handle the failure well. You can make sure that all old applications are upgraded first, but it can be easy to make a mistake and it doesn’t help if you are storing the protos in any datastore (even short-lived, like memcached). The same sort of situation applies when removing a required field.

Many required fields were “obviously” required until… they weren’t. Let’s say you have an id field for a Get method. That is obviously required. Except, later you might need to change the id from int to string, or int32 to int64. That requires adding a new muchBetterId field, and now you are left with the old id field that must be specified, but eventually is completely ignored.

When those two problems are combined, the number of beneficial required fields becomes limited and the camps argue over whether it still has value. The opponents of required weren’t necessarily against the idea, but its current form. Some suggested developing a more expressive validation library that could check required along with something more advanced like name.length > 10, while also making sure to have a better failure model.

Proto3 overall seems to favor simplicity, and required removal is simpler. But maybe more convincing, removing required made sense for proto3 when combined with other features, like removal of field presence for primitives and removal of overriding default values.

I’m not a protobuf developer and am in no way authoritative on the subject, but I still hope that the explanation is useful.

More Related Contents:

  1. How to define an optional field in protobuf 3
  2. protoc-gen-go-grpc: program not found or is not executable
  3. raw decoder for protobufs format
  4. How to create GRPC client directly from protobuf without compiling it into java code
  5. Getting Segmentation fault for implementing ByteSize() in C++ for protobuffer
  6. Unable to install gRPC for go in mac os
  7. Are there C++ equivalents for the Protocol Buffers delimited I/O functions in Java?
  8. InfogainLoss layer
  9. What does the protobuf text format look like?
  10. Google Protocol Buffers: JavaScript Example
  11. How to build Google’s protobuf in Windows using MinGW?
  12. TensorFlow saving into/loading a graph from a file
  13. Errors when linking to protobuf 3 on MS Visual C
  14. Retrieve large blob from Android sqlite database
  15. Biggest differences of Thrift vs Protocol Buffers? [closed]
  16. Undefined reference to google::protobuf::internal::empty_string_[abi:cxx11]
  17. How to solve “AttributeError: module ‘google.protobuf.descriptor’ has no attribute ‘_internal_create_key”?
  18. Testing a gRPC service
  19. NodeJs Error – Failed to load gRPC binary module because it was not installed for the current system Expected directory?
  20. C++ Serialization Performance
  21. What’s the best way to represent System.Decimal in Protocol Buffers?

Leave a Comment