Example-first Documentation

written by Zach Morrissey on 2019-07-07

When I read through your (library|module|framework)'s documentation I will skip everything and scan until I find the first example.

class Example:
    """
    An example class.
    """

    for i in range(100):
        # if you're like me, you will have read this prior to the opening sentence.
        print("hello world!")

The one unifying thing about many programmers is that they're driven towards problem-solving, and they're going to have a goal set when the finally reach your documentation. Documentation that aims to identify those goals and solve them as directly as possible is the best documentation. Users want the fastest way to prove and/or disprove their hypothesis.

The most direct way, you might imagine, is examples. Lots of examples. Everywhere. Nothing comes close to how useful actual usage is.

Two Common Ways I Use Documentation

  1. Checking how closely your examples match my intended use case. Everyone has a mental model of how they think a particular library will work prior to using it, and they will try to navigate to the right pages by searching for terms related to that. If this mental model is wrong, it's important to know why that is and provide examples for what the alternatives look like.
  2. Attempt to install your software in a variety of ways I already know, and fall back to reading the "Getting Started" page when that doesn't work.

More specifically, if users can't see the actual functions or classes they intend to use demonstrated, they will avoid your framework entirely.

Although you'll see more seasoned engineers throw around "RTFM" or other phrases extolling the virtues of painstakingly reading the manpages of a particular tool, but I will always have a shorter-term goal in mind than holistic understanding. Most documentation is intended as reference, but there still exists a large spectrum of user experience in navigating that reference information.

Reasons Whey They're Looking For Examples

Ternary JS operator describing how seniors feel about juniors' questions when they're not 100% well-researched.

  1. Starting off and want a frame of reference.
  2. Users know what an implementation might look like, but don't know what it's called.
  3. StackOverflow-inspired copy/paste-heavy workflow.
  4. Asking for help on the internet sucks.
  5. Asking for help from your coworkers can suck too.

If you believe it's easiest to get started using a framework if the examples are good, then it's truly the most user-focused way of helping your users out. It's the quickest route from intention to implementation.

Model Your Docs After Successful Software

There are an endless number of under-documented pieces of code out there, and their success or failure is largely driven around how quickly users can get at the answer that fits their mental model of how your library works.

Leaving some examples of great examples-driven documentation here:

An Architecture For Personal Data Engineering

written by Zach Morrissey on 2019-03-26

Performing in-depth data analyses on things in my own life have always been more difficult than in my working life, primarily because there I've normally had teams of engineers who have helped in setting up and maintaining most of the analytics-focused infrastructure that I've used. Supporting myself with the same sort of infrastructure I was used to was an adventure, one that started much earlier and has since grown from there.

Miscellaneous fake graphs for fun.

Scope

To most ends, downloading CSVs and plugging them into your spreadsheet software of choice will do most tasks well enough. For my purposes though, I wanted to scale this past what manual analysis was going to get me. There's a few key areas that I wanted this for:

  • Personal finances. Most of the data that I create finds it's way into financial systems that are hard to get your information out of. I'm not against the bank having my spending data, but I am frustrated that it's so hard for me to get it too.
  • Fantasy sports. I love sports and I love stats.
  • House shopping. This data is heaaaavily guarded and it's hard to find anything that isn't breaking some sort of ToS to get.

This doesn't necessarily create a system that requires dedicated data engineering work, but some of the goals that I had for it did. These were:

  • Automated. I'd like to see how repeatable and reproducible these analyses can be.
  • Modeled. I'd like to build and train some models related to how I live my life to see if there's any predictive benefit to these things.
  • Interactive. - For everything that you see here, I'd like to have some sort of interface that I can open, preferably via web browser.

The Bits n' Pieces

For doing this sort of work, I set up a few infrastructural components. Hardware is a low-spec server with Proxmox (for VMs) and Docker/Portainer.

Diagram of my personal data engineering architecture.

  • Postgres database - My primary datastore. This is the hub of all activity that I do, serving as both an application backend / transactional database as well as an analytical database. Each are neatly separated out into different schemas. I use DBeaver as a SQL client for ad-hoc querying, manipulation, setup, etc. Since a single Postgres instance can scale to vastly larger workloads than I'd ever be able to throw at it, this seemed like a good backbone for everything.
  • Apache Airflow - In it's simplest form, I'm using this basically as a scheduled job engine. Building on my earlier post about Airflow, I've expanded my usage of it to a significant variety of different DAGs. This uses the same Postgres database as earlier.
  • Jupyter Notebooks - Interactive analytical code/markdown files served up in your web browser. Jupyter is the ultimate tool in interactive analytical computing, with RStudio being the only major other option I considered (went with Jupyter, if for nothing else, because I find python more enjoyable to write). There's great options available if you want to run this on a server, I found that I like just running it locally the best.
  • Apache Superset - Web-based Dashboards. Superset, similar to Airflow, is based on Python/Flask and can be run in a single Docker container if you so desire. Since I need some way of sharing some of these results with a significant other (namely finance + house shopping data), I needed to have some sort of dashboarding software in order to have that visible to someone who will access it via web browser.

My Workflow

Since the pieces have been set in place now, this is my general workflow:

Process workflow from Airflow task to Jupyter notebook and then to Superset dashboard.

  1. Find a new datasource. Generally, this is something scraped from the web or pulled from a service I use.
  2. Write an Airflow task to pull that data on a regular schedule (normally nightly).
  3. In Postgres, create schema / table in order to store info.
  4. Use downstream in:
    1. Superset dashboard. This is normally when I've created a datasource that I'm interested in (i.e. personal spending data) and want to look at from a variety of angles.
    2. Jupyter notebook. Normally, this is to either create a more in-depth analysis (any modelling required) or to investigate one hunch / idea that I've got. In the event that I something useful to replicate, I ship this upstream into an Airflow task.
    3. Exported to some sort of printable / shareable thing. Good example for this is pre-draft research for fantasy sports.

Tips

This was a fun adventure to go on, but I'm happy that it's stabilized by this point.

  • It's fun stuff. It really is. Using data to analyze parts of your own life is something people rarely get to do.
  • There's a tradeoff between having a robust, feature-filled system and an easy one. As your needs change, your tools tend to change too. My aims for this outgrew my cron/jupyter/Google Sheets hackjob before trying to take on any of this.
  • Getting data is the hardest part. Significantly more time for me is spent writing data fetching tasks than anything else.

Building Dynamic Dungeons using Unity Tilemaps

written by Zach Morrissey on 2019-02-11

In one of the last projects I'm doing for school, I'm working on a game wherein everything in the game is procedurally generated using a dungeon-building algorithm.

Fresh Content Each Time

One of the main draws of building a Roguelike game is figuring out how to procedurally generate dungeons. While it's difficult to replicate the high-quality hand-crafted content of titles like Legend of Zelda: A Link to the Past, you can approximate a lot of the same experience while coming up with a new layout each time the game is loaded.

Achieving this is, as you may imagine, pretty difficult.

There are a lot of resources online, but most of them follow the same pattern - create a set of rooms, connect them, place items within them. There's a significant amount of wiggle room within those, but that's the basics.

The Dungeon of Deneb logo and intro scene

Unity Set Up

The dungeon I built is based on a Unity Tilemap, which is effective a grid of 1x1 squares that you can place tiles within. For the game that I'm working on, I made each tile 16x16 pixels (you can set them to the density that you're planning on making your tile sprites).

The unity objects involved:

  • Two tilemaps (Created by Game Object > 2D Object > Tilemap). Spawning a tilemap will create a Grid object as its parent. I created two Tilemaps and connected them to the same parent. I attached my Dungeon.cs building script as a component to this parent.
    • Floor tilemap - This is what I use to tile the floor background, and this is the main data for the dungeon layout (since walls are just created using the floor layout).
    • Walls (Collidable) tilemap - This one has a collider added to it, so the player can't walk through them.
  • Tile Palette - This contains all of the floor / wall tiles that I use to build the dungeon. It's nice to have one of these so you can manually paint some on if you want.

The "Dungeon Simplified" tile palette in the Unity Sidebar.

The Algorithm

Since dungeon generation algorithms can fall into a number of ways that they can end up in an infinite loop where they will never end up with a good dungeon layout, it's best to program each step of this in the following way:

  • While not at my limit of tries
  • Try
  • If failed, reset everything and increment how many tries I've done.

Steps

  1. Place a number of rooms in a grid.
  2. Derive a graph for the grid. The recommended way I found to do this was to create a Delauney Triangulation. This is somewhat complicated, so it's recommended to find an algorithm or package that will do this for you.
  3. From that graph, derive a minimum spanning tree. I found that this was easy enough to do using Prim's Algorithm, which is greedy and runs quickly.
  4. Verify that the graph meets your desired gameplay goals. For me, I wanted to make sure that I had at least two non-overlapping subtrees that comprised >25% of the nodes in the tree, so that I could use these as specific gameplay areas.
  5. Connect all the parent/child relationships in the tree with corridors.
  6. Once that's done, add some optional corridors for the nodes within the same subtree, so that you can remove the perfect "tree"-ness of the layout and add some variety to it.
  7. Choose a boss room. For mine, I simply chose a leaf node in the larger subtree with a higher-than-average floor area.
  8. Choose rooms to place keys in that will open locked doors.
  9. Lock the doors.
  10. Place items and enemies within those rooms according to some sort of sliding difficulty scale. You can easily derive this difficulty on a number of simple heuristics (i.e. bigger room = more enemies, further depth in tree = harder enemies, etc).

Go!

Layouts Generated

First, a key, so you can understand what you're looking at:

  • Rooms:
    • Green / White = starting room / starting area
    • Red = boss room
    • Light blue = harder-than-starter area, Dark Blue = advanced area
    • Yellow = room with a key
  • Corridors:
    • White - Parent/child relationship in tree
    • Yellow - Optional corridor added between two rooms in the same subtree.

Dungeon Layout 1

The key to the light blue area spawns in the green / white area, the key to the dark blue area spawns in the light blue area. The key to the boss room is placed within the dark blue area.

Dungeon Layout 2

As you can see in this one, sometimes the room with a key in it spawns directly next to the boss (in the first example above), but sometimes it's located fairly far away in a different corner (in this example, on the bottom left).

Dungeon Layout 3

In this example, I've included the minimum spanning tree generated during the dungeon building process. If you note the yellow corridors between rooms, those are randomly added between rooms within the same subtree.

Applying Tiles

The TileMap.SetTile function can then be used to programmatically fill out which tiles are supposed to go where. Once you've got the room data figured out, it's pretty easy to determine where to set tiles (floor tiles set within any room or corridor, wall tiles set in any empty space that borders a corridor or room).

The result:

Dungeon Layout with tiles applied to the TileMap

From here, you can generate the items and enemies how you see fit.

Switching Careers to Software Engineering: Six Months Update

written by Zach Morrissey on 2019-01-07

Lessons Learned

Now that I've been in my role for a while, it's time to reflect on that period and try to put it in context of being in a fully different role roughly 1 year ago.

The skinny:

  • I don't regret it at all.
  • There's a lot I don't know.

...and that's basically it. I'm learning a lot every day. If you were expecting some sort of sea change in who you are after you get the job, it's likely not going to be there. It's a new set of challenges.

You're Good Enough

The thing that's taken me aback most of all upon switching to a full time software engineering role was how comfortable it feels. There's a lot of practices I held near and dear to my heart on a personal level that have translated well into an engineering practice.

My takeaways:

  • You're likely a good enough engineer.
  • You probably know more about engineering as a job than you think that you do.
  • Your teammates are happiest when you're resourceful. Research your questions before bugging people. Look for novel solutions. Use what's available to you.
  • It doesn't hurt to be open about what you're looking for / dealing with.
  • Don't be too entrenched in your opinions.
  • The real world of engineering is different than whatever source of information you're reading about online.

Don't Expect Things to be Easy

Like any job, software engineering comes with its own set of frustrations. You're likely going to have to use software you don't want to. You're probably going to have to revive long-dead code that's uncommented and unloved. You may even use unsexy languages / frameworks / approaches.

That's well-known enough that it should've been obvious, but until you're there it's much too easy to write things off.

I've noticed the things that I struggle with the most aren't even technical challenges. Sure, those are there. The majority of the issues, however, come from trying to be productive as a team. Solving things not just for myself but for the entirety of the organization. If I find a script that helps me debug something, that's great, but it may only help me. Whereas a little bash script you use on your own machine may have been helpful before, there's a lot of added difficulty if you want to share it with others, automate it, or make it a standard practice.

Motivating and working with groups of people is not an easy practice, and there will always be discrepancies in what people believe is the best approach.

Don't Stop Learning

Software engineering is a role where you can find yourself in a life of learning. Finding the job doesn't mean that part is over, it means you've just passed the first level.

There's a lot you can do in your spare time, but the best thing you can do is occassionally challenge yourself to learn something new and try a different technology. I've found that the times I'm most productive are when I can recognized a pattern in one approach based on something I've done somewhere else. It's a satisfying thing to see your curiosity rewarded, doubly so in programming.

The obvious suggestions:

  • Take a course on Coursera, Udemy, etc.
  • Build a personal project.
  • Try to see if you can improve something you wrote in the past in a new framework / language / paradigm.
  • Benchmark something and see if you can make it faster.
  • Refactor an old set of code for maintainability. Make it easier for others.

This is not comprehensive, nor does it fit everyone, but it should give you an idea of things you can do that aren't just the "Get ticket, do ticket" type engineering it's easy to fall into. People love to see you excited about things, and being passionate will make you happier.

What's next?

Frankly, I don't know. I'm optimistic.

I'll follow this up with another check-in at the one year mark.

Switching Careers to Software Engineering: A Retrospective

written by Zach Morrissey on 2018-08-01

I've just been hired into my first dedicated software engineering role. After returning to school for CS in November 2016, I've just emerged on the other side almost two years later. This is the first thing I've done in my career that feels like it fits how I think and work, and I'm planning on making the most of it.

My approach:

  • Move into a semi-technical role where I had better access to engineering (TPM / BI Analyst).
  • Start a CS degree. I opted for the Oregon State Online Post-bacc in CS, which is an excellent program that I highly recommend.
  • Work up a sizable resume of personal projects that prove that you can do the work (I'm still working on this bit. This blog is one of those).

And it I'm glad to say it worked!

Three Important Aspects of Switching into Software Development

Here's the different angles I took on understanding the career switch as I went through it. It's an odd one, but depending on where you're transitioning from, it can be an excellent choice.

The Learning Aspects

Compared to many types of career you'd try to switch into, many people switching into CS will be touching a STEM-type field for the first time in their lives. Comparably, it's one of the more accessible sciences to get into because computers are ubiquitous and drive everything in the world today. I've spent a lot of the last two years just learning as much as I could.

  • First and most obviously, it's hard. Few people can do it quickly. It takes most people a few years at least. When you hear stories about people who were grinding Leetcode for 3 months and got the offer, know that they're vastly outside the normal distribution of how long it takes people.

  • The problem domain is vast. You quickly get into territory for which there is no set solution, everything has a trade-off, and it's often difficult to espouse why. This is a significant generalization, but the breadth of problems that I've had to solve in my short time doing software engineering vastly eclipses most roles I've had or worked with previously. There's so much to know, and you'll likely never be an expert in more than a few small subdomains of the large problem space.

  • There are few, if any, scenarios today where you'll happen to fall into software engineering without significant, dedicated effort. It's common for senior engineers to mention how they may have done this back in the day when they got their start programming for Amiga, Commodore 64, or in the early days of the web. This represents a minority of those career switching into development.

  • There are plenty of ways to get the education for it. Bias showing, I recommend spending the time and money to go to school through an accredited college/university. Boot camps are enticing in that they're quick turnarounds, can get you very career-ready compared to school, and are generally much less expensive. In general, I'd be cautious of the drastic uptick in Boot Camps that have sprung up as of late. There are plenty of excellent ones, but there are also enough bad ones to offset those. You may find yourself at a disadvantage for any job that requires a solid foundation in data structures/algorithms/operating systems/etc. Boot camps generally focus very heavily on web development, and usually teach a single ecosystem (i.e. Node/Express, Ruby on Rails, etc).

The Technical Aspects

There's a lot of different tech out there and you're not going to learn it all. There's likely a lot of stuff you'll never touch in your lifetime. It's a very wide field with tons of specializations, most of which you'll probably have limited ability to gain meaningful experience in school or boot camp or being self-taught. Seeking out which technologies you want to learn/use is good, but with some caveats:

  • As a junior, being opinionated about which technologies you want to use will limit your opportunities. If you don't particularly like Microsoft, or don't like Java, you may cut yourself off from gaining experience. You can be selective if you so desire, but it will restrict your career in the short-term. Getting that sexy startup job with 2-years experience in any sort of software development role will be significantly easier than having none.

  • The single most advantageous skill is being resourceful and looking for answers. Good engineers are the ones who can use the resources available too them (Google, stackoverflow, Github, etc) and make decent, working code out of it.

  • Know at least one low-level or compiled language and at least one high-level or interpreted language. The concepts from the lower level languages you learn will be applicable to using the high-level language. Someone using a high-level language may never know why something like copying an array is slow, because they're not aware of how memory works in relation to the code you're writing. Writing C or a similar low-level language quickly eradicates many common misunderstandings.

  • Buy/build a home server or get an AWS account. Start tinkering as a way of learning. Set up some services and have fun.

  • Learn databases and SQL. Most applications you use will interact with a database in some manner. It's easy to get proficient.

  • Get good at setting up your working environment. Having a great local setup using tools you're comfortable with pays significant productivity dividends. There's a reason that developers are so choosy about which tools they like.

The Human Aspects

  • Managers outside of development won't help your career switch. If you're working on improving your skills to the point where you could start as a developer (by school, code boot camp, etc), they will often espouse support but generally have no idea what that means they should do to help you. Doesn't mean you shouldn't be honest with them about your intentions, but don't expect them to come to you with solutions.

  • Mentorship is exceedingly difficult to come by. You must purposefully seek it out if you want it. If you end up having an opportunity with someone who shares an interest in your future, nurture that flame. Let them know you're interested. It's easy to feel defensive about being inexperienced. This is a nice way to gain some perspective on what that means for you right now.

  • When you hear about software engineer hiring shortages, they're referencing senior engineering talent. Junior roles can often be saturated with new listings all the time, and suffer from a glut of genuinely unqualified applicants. That means people who list things on their resume they have zero experience with, to the point they haven't even done a Udemy tutorial on them.

  • There's a lot of noise out there. Many people, with many opinions (including this one), and lots of people will tell you the best approach. If there's only one thing to keep in mind, it's to be patient with yourself and understand that nobody else is in your exact scenario.

Navigating the Transition

Switching into software development these days can feel like jumping on a bandwagon where everyone is smarter than you, paid better than you, and has intense opinions on what is right and wrong. It can feel like an insurmountable hill to climb.

However, there are plenty out there like myself for which software engineering is an open door to a much better career. Once I've been in the role for a while, I'll have a part two to this post where I cover my transition period.