Getting Started with Envoy HTTP Filter in Rust

Let’s create an Envoy HTTP Filter in Rust using GetEnvoy Extension Toolkit.

1. Pre-requirements

  1. πŸ’» Install GetEnvoy.

    βœ… check:

    $ getenvoy --version
        
    getenvoy version 0.2.0
    
  2. πŸ’» Install Docker.

    βœ… check:

    $ docker --version
        
    Docker version 19.03.8, build afacb8b
    

2. Scaffold a new HTTP Filter extension

πŸ’» To walk through the interactive wizard, run:

$ getenvoy extension init

πŸ’‘ Alternatively, to skip the wizard, provide the arguments on the command line, e.g.:

$ getenvoy extension init \
    --category envoy.filters.http \
    --language rust \
    --name me.filters.http.my_http_filter \
    my_http_filter

βœ… getenvoy will scaffold a new extension workspace in the target directory:

$ tree -a my_http_filter
 
my_http_filter
β”œβ”€β”€ .cargo
β”‚   └── config
β”œβ”€β”€ .getenvoy
β”‚   └── extension
β”‚       └── extension.yaml
β”œβ”€β”€ .gitignore
β”œβ”€β”€ Cargo.toml
β”œβ”€β”€ README.md
β”œβ”€β”€ src
β”‚   β”œβ”€β”€ config.rs
β”‚   β”œβ”€β”€ factory.rs
β”‚   β”œβ”€β”€ filter.rs
β”‚   β”œβ”€β”€ lib.rs
β”‚   └── stats.rs
└── wasm
    └── module
        β”œβ”€β”€ Cargo.toml
        └── src
            └── lib.rs

πŸ“šFor more details see the Extension Toolkit Walkthrough.

3. Build the extension

πŸ’» To build a WebAssembly module (*.wasm file) loadable by Envoy, run:

$ getenvoy extension build

βœ… getenvoy will run cargo build to generate a WebAssembly module (extension.wasm):

$ tree target/getenvoy

target/getenvoy
└── extension.wasm

πŸ“šFor more details see the Extension Toolkit Walkthrough.

4. Run unit tests

πŸ’» To execute Rust unit tests, run:

$ getenvoy extension test

βœ… getenvoy will run cargo test:

running 1 test
test tests::should_initialize ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

πŸ“šFor more details see the Extension Toolkit Walkthrough.

5. Run the extension in Envoy (the “hard” way)

πŸ’‘ To get a better understanding of what actions must take place in order to run the extension, let’s manually reproduce steps of getenvoy:

  1. πŸ’» First, we need to build a WebAssembly module (*.wasm file) that will be loaded by Envoy:

     $ getenvoy extension build
    

    βœ… getenvoy will run cargo build to generate a WebAssembly module (extension.wasm):

     $ tree target/getenvoy
         
     target/getenvoy
     └── extension.wasm
    
  2. πŸ’» Next, we need to prepare Envoy config suitable for this particular extension:

     $ getenvoy extension examples add
    

    βœ… getenvoy will generate the default example setup:

     $ tree .getenvoy/extension/examples/
         
     .getenvoy/extension/examples/
     └── default
         β”œβ”€β”€ README.md
         β”œβ”€β”€ envoy.tmpl.yaml
         β”œβ”€β”€ example.yaml
         └── extension.json
    
  3. πŸ’» Next, we need to find out what version of Envoy the extension is compatible with:

     $ cat .getenvoy/extension/extension.yaml
    

    βœ… getenvoy records the compatible version of Envoy at a time of getenvoy extension init:

     # Runtime the extension is being developed against.
     runtime:
       envoy:
         version: wasm:1.15
    
  4. πŸ’» Next, we need to download Envoy binary of that version:

     $ getenvoy fetch wasm:1.15
    

    βœ… getenvoy will download Envoy binary and cache it under $HOME/.getenvoy:

     $ tree $HOME/.getenvoy/builds
         
     $HOME/.getenvoy/builds
     └── wasm
      Β Β  └── 1.15
      Β Β      └── darwin
      Β Β          └── bin
      Β Β              └── envoy
    
  5. πŸ’» Finally, we need to wire together various configuration bits (such as, Envoy config, *.wasm file,
    extension-specific config) and start the Envoy process:

    $ getenvoy extension run
    

    βœ… getenvoy will generate the actual Envoy config (by resolving placeholders in the configuration template ) and start the Envoy process in the foreground:

    info	Envoy command: [$HOME/.getenvoy/builds/wasm/1.15/darwin/bin/envoy -c /tmp/getenvoy_extension_run732371719/envoy.tmpl. yaml]
    ...
    [info][main] [external/envoy/source/server/server.cc:339] admin address: 127.0.0.1:9901
    ...
    [info][config] [external/envoy/source/server/listener_manager_impl.cc:700] all dependencies initialized. starting workers
    [info][main] [external/envoy/source/server/server.cc:575] starting main dispatch loop
    

πŸ“šFor more details see the Extension Toolkit Walkthrough.

5+. Run the extension in Envoy (the “easy” way)

πŸ’» To run the extension in Envoy, execute:

$ getenvoy extension run

βœ… getenvoy will download Envoy binary, generate a sample Envoy config and start the Envoy process in the foreground:

info	Envoy command: [$HOME/.getenvoy/builds/wasm/1.15/darwin/bin/envoy -c /tmp/getenvoy_extension_run732371719/envoy.tmpl.yaml]
...
[info][main] [external/envoy/source/server/server.cc:339] admin address: 127.0.0.1:9901
...
[info][config] [external/envoy/source/server/listener_manager_impl.cc:700] all dependencies initialized. starting workers
[info][main] [external/envoy/source/server/server.cc:575] starting main dispatch loop

πŸ“šFor more details see the Extension Toolkit Walkthrough.

6. Make sample requests

πŸ’» Checkout README.md file included in the default example setup:

   $ open .getenvoy/extension/examples/default/README.md

βœ… In the case of a HTTP Filter, instructions will be similar to the following:

   ## How to use
   
   1. Make HTTP request
      ```shell
      curl http://0.0.0.0:10000
      ```
   2. Checkout `Envoy` stdout

πŸ’» Follow How to use instructions from README.md

βœ… HTTP Filter extension will log the following to the output of Envoy process:

my_http_filter: #2 new http exchange starts at 2020-07-01T18:22:51.623813+00:00 with config:
my_http_filter: #2 observing request headers
my_http_filter: #2 -> :authority: 0.0.0.0:10000
my_http_filter: #2 -> :path: /
my_http_filter: #2 -> :method: GET
my_http_filter: #2 -> user-agent: curl/7.64.1
my_http_filter: #2 -> accept: */*
my_http_filter: #2 -> x-forwarded-proto: http
my_http_filter: #2 -> x-request-id: 8902ca62-75a7-40e7-9b2e-cd7dc983b091
my_http_filter: #2 http exchange complete

πŸ“šFor more details see the Extension Toolkit Walkthrough.

7. Add a new feature

πŸ’‘ To get familiar with the source code, refer to Extension Toolkit Reference.

πŸ’‘ Let’s add a new feature to the extension - inject an extra header into proxied HTTP responses.

  1. πŸ’» First, let’s update extension config to hold the name of a header to inject (added lines are marked with // added code):

    src/config.rs

    /// Configuration for a Sample HTTP Filter.
    #[derive(Debug, Default, Deserialize)]
    pub struct SampleHttpFilterConfig {
       #[serde(default)]                 // added code
       pub response_header_name: String, // added code
    }
    
  2. πŸ’» Next, let’s add on_response_headers method to the SampleHttpFilter (all lines need to be added):

    src/filter.rs

    /// Called when HTTP response headers have been received.
    ///
    /// Use `filter_ops` to access and mutate response headers.
    fn on_response_headers(
        &mut self,
        _num_headers: usize,
        _end_of_stream: bool,
        filter_ops: &dyn http::ResponseHeadersOps,
    ) -> Result<http::FilterHeadersStatus> {
        if !self.config.response_header_name.is_empty() {
            filter_ops.set_response_header(
                &self.config.response_header_name,
                "injected by WebAssembly extension",
            )?;
        }
        Ok(http::FilterHeadersStatus::Continue)
    }
    
  3. πŸ’» Finally, let’s update extension configuration in the default example setup (all lines need to be added):

    .getenvoy/extension/examples/default/extension.json

    {
     "response_header_name": "my-header"
    }
    
  4. πŸ’» To verify the changes, run:

    $ getenvoy extension run
    

    Then, in a separate shell, run:

    $ curl -i localhost:10000
    

    βœ… you should receive a response similar to:

    HTTP/1.1 200 OK
    content-length: 22
    content-type: text/plain
    date: Tue, 07 Jul 2020 18:36:23 GMT
    server: envoy
    x-envoy-upstream-service-time: 0
    my-header: injected by WebAssembly extension
         
    Hi from mock service!
    

    πŸŽ‰ Notice my-header: injected by WebAssembly extension that was injected by the HTTP Filter extension.

8. Add a new metric

πŸ’‘ To get familiar with the source code, refer to Extension Toolkit Reference.

πŸ’‘ Let’s update the extension to expose metrics about its new behaviour.

Specifically, let’s provide a counter with a number of HTTP responses the extra header has been injected to.

  1. πŸ’» First, let’s add a new counter to the Extension Stats (added lines are marked with // added code):

    src/stats.rs

    use envoy::host::stats::Counter;
    
    // Sample stats.
    pub struct SampleHttpFilterStats {
       requests_total: Box<dyn Counter>,
       responses_injected_total: Box<dyn Counter>,              // added code
    }
         
    impl SampleHttpFilterStats {
       pub fn new(
           requests_total: Box<dyn Counter>,
           responses_injected_total: Box<dyn Counter>,          // added code
       ) -> Self {
           SampleHttpFilterStats {
               requests_total,
               responses_injected_total,                        // added code
           }
       }
       pub fn requests_total(&self) -> &dyn Counter {
           &*self.requests_total
       }
       pub fn responses_injected_total(&self) -> &dyn Counter { // added code
           &*self.responses_injected_total                      // added code
       }                                                        // added code
    }
    
  2. πŸ’» Next, let’s create the counter at a time when Extension Factory is created (added lines are marked with // added code):

    src/factory.rs

    /// Creates a new factory.
    pub fn new(clock: &'a dyn Clock, stats: &dyn Stats) -> Result<Self> {
        let stats = SampleHttpFilterStats::new(
            stats.counter("examples.http_filter.requests_total")?,
            stats.counter("examples.http_filter.responses_injected_total")?, // added code
        );
        // Inject dependencies on Envoy host APIs
        Ok(SampleHttpFilterFactory {
            config: Rc::new(SampleHttpFilterConfig::default()),
            stats: Rc::new(stats),
            clock,
        })
    }
    
  3. πŸ’» Finally, let’s increase the counter in the on_response_headers method of the SampleHttpFilter (added lines are marked with // added code):

    src/filter.rs

    /// Called when HTTP response headers have been received.
    ///
    /// Use `filter_ops` to access and mutate response headers.
    fn on_response_headers(
        &mut self,
        _num_headers: usize,
        _end_of_stream: bool,
        filter_ops: &dyn http::ResponseHeadersOps,
    ) -> Result<http::FilterHeadersStatus> {
        if !self.config.response_header_name.is_empty() {
            filter_ops.set_response_header(&self.config.response_header_name, "injected by WebAssembly extension")?;
            self.stats.responses_injected_total().inc()?; // added code
        }
        Ok(http::FilterHeadersStatus::Continue)
    }
    
  4. πŸ’» To verify the changes, run:

    $ getenvoy extension run
    

    Then, in a separate shell, run:

    $ curl -i localhost:10000
    $ curl -i localhost:10000
    
    $ curl -s localhost:9901/stats | grep responses_injected_total
    

    βœ… you should get the output similar to:

    examples.http_filter.responses_injected_total: 2
    

    πŸŽ‰ Our new counter gets incremented on every proxied HTTP request.

That concludes a brief look into Envoy extension development using GetEnvoy Extension Toolkit.

9. What’s next

πŸ’‘ Use the following resources to find more about GetEnvoy Extension Toolkit: