Many applications rely on gRPC to connect services, but a number of modern load balancers still do not support HTTP/2, and, in turn, gRPC. In an earlier blog post, we showed a way to take advantage of the gRPC-Web protocol to circumvent this issue. That solution works well for non-client-streaming gRPC calls — with this new approach, we can support client/bidirectional-streams.
In our earlier writing, we briefly mentioned that WebSockets may actually help us resolve our client/bidi-streaming problem. Since we published that article, we have been able to implement a solution using WebSockets, which we discuss here.
gRPC over WebSocket
The WebSocket protocol is ideal for our needs: it is HTTP/1.x compatible, supported by many modern load balancers, and client/bidi-streaming capable.
Luckily, a comprehensive specification is available for the gRPC protocol, so we have been able to transcode gRPC requests/responses into WebSocket messages without any guesswork. The workflow is as follows:
- The client initiates a gRPC request to the server
- The client initiates a WebSocket connection with the server
- The server accepts the WebSocket connection
- The client transcodes the gRPC request on the fly and sends it via the WebSocket connection
- The server reads the request off the WebSocket connection and responds via the same connection
- The client reads the response off the WebSocket connection
- The server closes the WebSocket connection upon completion
Similar to our gRPC-Web “downgrade” implementation, we opt for spawning a local HTTP/2 client-side proxy to handle the transcoding and WebSocket connection. While this approach does add an extra hop in the network communication (albeit via a local in-memory pipe, net.Pipe
), it is less invasive than modifying gRPC client-library code. We also opt for using a modified server handler which does not add an extra hop in the network communication. An example workflow follows:
Transcoding Requests
The gRPC protocol defines a request as a stream of bytes defined as follows in ABNF form (note: we will show only the subset of definitions required for this discussion):
Request -> Request-Headers *Length-Prefixed-Message EOS
Length-Prefixed-Message -> Compressed-Flag Message-Length Message
Compressed-Flag -> 0 / 1 # encoded as a 1-byte unsigned int
Message-Length -> {message length} # 4-byte big-endian unsigned int
We won’t worry about the Request-Headers
, as they remain untouched and are simply forwarded along the wire with the initial WebSocket connection request. Instead, we will focus on Length-Prefixed-Message
, which is the request’s body.
We cannot wait and send the entire series of length-prefixed messages as one payload, as the request may be client-streaming. So, we opt for a different approach: send a new WebSocket message one Length-Prefixed-Message
at a time. We know how long each message is, as it is length-prefixed, so we may just simply forward each message along the WebSocket connection.
(Side note: we can rely on intermediate proxies to respect WebSocket message boundaries and to not buffer individual WebSocket messages. A WebSocket message can be split up into multiple WebSocket frames, but that is an implementation detail.)
The last concern is to find a way to signal the server that we are done sending messages. The gRPC protocol handles this step by setting the HTTP/2 END_STREAM
flag on the final HTTP/2 data frame. However, Golang’s HTTP/2 library does not give us access to low-level constructs such as HTTP flags or any other part of the HTTP/2 framing. To signal completion, we take inspiration from the gRPC-Web protocol.
Recall that a gRPC request is simply a sequence of bytes, and the body is a series of Length-Prefixed-Messages
. The first byte of each Length-Prefixed-Message
represents the Compressed-Flag
, which will be set to either 0
or 1
. This construct leaves us with seven unused bits.
The gRPC-Web protocol uses the most-significant bit (MSB) in the Compressed-Flag
byte to indicate that a message contains trailing metadata. We adopt the same approach for the data sent by the client: the final message sent by the client will always have the MSB set. Since the client does not send any trailing metadata, the End-of-Stream (EOS) message is uniquely identified by having the MSB in the first byte set and a Message-Length
of zero.
Transcoding Responses
The gRPC protocol defines a response as follows in ABNF form (note: we will show only the subset of definitions required for this discussion):
Response -> (Response-Headers *Length-Prefixed-Message Trailers) / Trailers-Only
Trailers -> Status [Status-Message] *Custom-Metadata
Trailers-Only -> HTTP-Status Content-Type Trailers
The process for transcoding responses is actually very similar to transcoding requests. However, instead of sending the headers as normal HTTP request headers, we send all bytes of the response as part of the WebSocket stream. The WebSocket protocol is a handshake-based protocol, so the actual HTTP response headers, which initiate the WebSocket connection, are sent immediately upon connection creation. The gRPC server may delay sending header metadata arbitrarily, so the header metadata must be sent via the stream.
Now we have to deal with Trailers, as the Length-Prefixed-Message
is the same as before. Trailers/Trailing metadata is actually handled in exactly the same way as the gRPC-Web protocol — sent as a normal response message with the MSB set in the leading byte — with the sole difference that the message is, of course, sent via a WebSocket stream, not as part of a normal HTTP response body.
(Side note: Again, we can rely on intermediate proxies to respect WebSocket message boundaries and to not buffer individual WebSocket messages, as some proxies may do with chunked HTTP/1.x responses.)
Updates to our Open-Source Repository
We recently announced we open-sourced our gRPC via HTTP/1 repository, and we have since updated the repository with the new functionality presented here.
Using the Library in Your Client Code
As mentioned in the previous post, we export a function ConnectViaProxy
, which replaces the typical grpc.DialContext
. The function header is the same as before, but we have added a new ConnectOption
, UseWebSocket
. When set, the library will use WebSockets to connect to the server; otherwise, we default to the gRPC-Web “downgrades.”
(Side note: we introduced two ConnectOptions
previously: ForceHTTP2
and ExtraH2ALPNs
. These options are ignored when WebSockets are enabled.)
An example usage is:
ctx := context.Background()
targetAddr := "https://my.example.com"
tlsClientConf := &tls.Config{}
// Traditional, non-proxy client
// cc, _ := grpc.DialContext(ctx, targetAddr, grpc.WithTransportCredentials(credentials.NewTLS(tlsClientConf))...)
// With the proxy client
cc, _ := client.ConnectViaProxy(ctx, targetAddr, tlsClientConf, client.UseWebSocket(true)...)
echoClient := echo.NewEchoClient(cc)
...
Why not always use WebSockets?
Though we could use WebSockets for all types of gRPC workflows (streaming and non-streaming), WebSockets come with their own baggage: they require an initial handshake, which adds latency, and they are not compatible with a standard gRPC server.
We leave choosing between the two approaches — gRPC-Web “downgrades” and WebSockets — to the user. We recommend using the gRPC-Web “downgrades” if the only requirement is to support unary requests, and we recommend using WebSockets otherwise. The gRPC-Web “downgrade” solution does support server-streaming requests, since HTTP/1.x supports it; however, it is possible an intermediate proxy could choose to buffer chunked responses, thereby not supporting server-streams.
(Side note: choosing the WebSockets approach means WebSockets will be used even when an intermediate HTTP/2 incompatible proxy is not present. The gRPC-Web “downgrade” solution takes an adaptive approach, and it will only downgrade if an intermediate incompatible proxy exists.)
In the future, we would like to have auto-sensing logic to detect the presence of a proxy and whether it supports server-streaming — then we will be able to automatically determine which solution to use for a given request.
Using the Library in Your Server Code
With these updates, the CreateDowngradingHandler
signature needs no changes. However, it’s important to note that if the user wishes to use WebSockets, then the downgrading handler must be used, as traditional gRPC servers do not support WebSockets. Users are still not required to use the downgrading server handler when the client library does not use WebSockets.
An example usage is:
grpcSrv := grpc.NewServer()
echo.RegisterEchoServer(grpcSrv, echoService{})
// 0 means the port is chosen for us.
lis, _ := net.Listen("tcp", "127.0.0.1:0")
// Traditional, non-downgrading-capable gRPC server
// grpcSrv.Serve(lis)
// With downgrading-capable gRPC server, which can also handle HTTP.
downgradingSrv := &http.Server{}
var h2Srv http2.Server
_ = http2.ConfigureServer(downgradingSrv, &h2Srv)
downgradingSrv.Handler = h2c.NewHandler(server.CreateDowngradingHandler(grpcSrv, http.NotFoundHandler()))
downgradingSrv.Serve(lis)
Conclusion
If you find yourself wanting to use gRPC in an environment where only HTTP/1.x traffic is allowed, you no longer need to fret! We have added our WebSockets solution to our already open-sourced solution to expand support to client/bidirectional streaming, and we welcome users and contributors.
About the author
Browse by channel
Automation
The latest on IT automation for tech, teams, and environments
Artificial intelligence
Updates on the platforms that free customers to run AI workloads anywhere
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
The latest on how we reduce risks across environments and technologies
Edge computing
Updates on the platforms that simplify operations at the edge
Infrastructure
The latest on the world’s leading enterprise Linux platform
Applications
Inside our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Customer support
- Developer resources
- Find a partner
- Red Hat Ecosystem Catalog
- Red Hat value calculator
- Documentation
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit