AWS, Software Development Engineer
April 14, 2014 - July 21, 2023
Lead the development experience of Java at Amazon, develop foundational software used by 80k builders, and 575k software projects. Maintain enterprise-level code base ranging from legacy to greenfield.
Incident Response for Amazon/AWS Enterprise Infrastructure
For 3 years (2020-2023) I provided incident response to critical remote code execution (RCE) security vulnerabilities for Amazon, including "Log4shell" (CVE-2021-44228) and "Spring4shell" (CVE-2022-22965) as well as others like Python's string formatting CVE (CVE-2021-3177). I jumped into foreign code bases and made upgrades or patches, ensured the change was delivered to >575k Amazon software projects, and provided support to other oncall engineers.
Roles and technologies
Backend Engineer, Frontend Engineer, Full-Stack Engineer, Python, Java, Spring Framework, Cyber Security, Application Security, Information Security, Gradle, Ant, Log4i, Maven
ARM support for Amazon proprietary build fleet
I created a Linux build image (Docker/OCI) for Amazon developers to perform basic development in a Linux ARM environment (AWS Graviton, an AArch64 compute architecture for Amazon EC2) that previously only supported Linux x86_64 architecture. The image included various common build tools and package management technologies common in Amazon/AWS like the OpenJDK, Node.js/NPM, Python, and CMake/GCC/Make/Ninja fully configured and tested. It also included some special-purpose software like Lua, Fortran, and GNU autotools.
While I performed the project by myself, I also contributed code to 5 other teams in Amazon's Builder Tools organization. This involved changes in build and local development CLIs written in Ruby and Java, changes in various build orchestration software written in Java, Go, and Perl. There were incidental changes to enable delivery of the completed build image to an Elastic Container Repository (Amazon ECR) via RSpec. I also ensured a complete story for provenance (a proprietary bill of materials is created for each build, accounting for idiomatic dependencies used) and ensured the build fleet team provisioned capacity and had proper scaling and telemetry.
Roles and technologies
Backend Engineer, DevOps Engineer, Full-Stack Engineer, Project Manager, Quality Assurance, Node.js, Python, Java, Go, Cloud Architecture, Linux, Docker, Amazon EC2, Perl, ARM, CMake, Fortran, gcc, Lua, Make, NPM, Rspec
Apache Maven wrapper for Amazon/AWS proprietary build system
I provided Apache Maven support via bootstrapping bash scripts that define an API required by build orchestration and emit the necessary configuration files for development to work without wrappers to support other use cases like IDE development (Tested with IntelliJ, VSCode, Eclipse, and in honor of James Gosling, also NetBeans).
I also wrote the documentation, developed example software using the wrapper, and templates for Amazonians to get started fast. The wrapper is capable of running builds and dependency resolution in a network-isolated build fleet. The solution used AWS CodeArtifact as a maven proxy, allows dependency resolution and publishing, and reports all dependency usage per build task for legal and security tracking. The proprietary build system is similar to technologies like Bazel, Buck 2, Nix, or Yocto/OpenEmbedded.
Roles and technologies
Backend Engineer, DevOps Engineer, Full-Stack Engineer, Java, Bash, Technical Writing, Maven
Apache Ant and Ivy wrapper for Amazon proprietary build system
I provided Apache Ant and Ivy support via bootstrapping bash scripts that deline an API required by build orchestration and emit the necessary configuration files for development to work without wrappers to support other use cases like IDE development.
I also wrote documentation, and developed an example library and application using the wrapper.
The wrapper is capable of running maven dependency resolution and builds in a network-isolated build fleet. The solution used AWS CodeArtifact as a maven proxy, allows dependency resolution and publishing, and reports all dependency usage per build task for legal and security tracking.
Roles and technologies
Backend Engineer, DevOps Engineer, Full-Stack Engineer, Java, Bash, Ant, Maven
Gradle support in Amazon's legacy and modern build systems
Between 2020 and 2023 I provided support and maintenance for Amazon's Gradle integrations to its legacy build system. The legacy build system resembles Nix/Hydra with a concept of always performing cascading builds of all Amazon/AWS software, the modern build system resembles Bazel/Buck/Nix but publishing to idiomatic repository proxies that leads to a more decentralized experience.
I ensured timely imports of Gradle and updates to our bootstrapping bash scripts and Gradle plugins. I also wrote technical migration guides for other Amazonians when they perform major version updates, and migrated older build scripts to use "Task Configuration Avoidance" APIs. I also developed and provided examples and templates with both Groovy and Kotlin build scripts for common use cases including a example applications in Java/Kotlin/Scala/Clojure/C++, an example Android application, and an example AWS Lambda Function.
Roles and technologies
Backend Engineer, DevOps Engineer, Frontend Engineer, Full-Stack Engineer, Product Manager, Java, Android, C++, Kotlin, Clojure, Scala, Technical
Design and implement cutover management web service (and clients) in large Amazon migration project
From June 2019 to June 2020 I designed and implemented a microservice with ~Oms overhead for clients, which managed cutover events while AWS's Commerce Platform (organization of many finance-related products in Amazon, about 500 engineers at the time) performed a migration of a subset of critical customer network traffic and data from a region in the US to a region in China.
The service design and database design was for an AWS Lambda based service, implemented in Java, backed by a DynamoDB NoSQL database, which allowed for >10ms latency on all operations. Because the service was required to operate in multiple AWS partitions (which means different Amazon IAM stacks) there was also design required for secure credential management.
I also provided clients (generated from Smithy-generated Swagger/OpenAPI bindings) for calling the service and bindings to an Amazon proprietary server framework that allowed for caching and side-loading configuration which lead to ~Oms overhead for clients.
For non-programmatic use cases I provided a CLI implemented in Python for the CRUD actions on creating cutover routes, wrote a bash script for bootstrapping and installing the CLT, and wrote technical instructions for installing and using the CLI.
Roles and technologies
Backend Engineer, Database Engineer, Tech Lead, Python, Java, APIs, Microservices, Database Architecture, Database Design, Web Services, Bash, Amazon Lambda, NoSQL, DynamoDB, Amazon IAM, OpenAPI, Swagger
More
I did more things, and will write about them soon.