Month: May 2018

Recently, I started reading a new book called Building Microservices: Designing Fine-Grained Systems, by Sam Newman. So far, the book is good. The first chapter gave a good overview of what microservices are. One of the key takeaways are about how you want to keep services small, with a core benefit being that you can iterate faster than you otherwise would have. However, in my experience, I have found that I (and others) can tend towards the reverse. That is, engineers might tend towards adding new code to existing monolithic services instead of adding code to new services. Funnily enough, we agree with the theory of microservices, but disagree with it in practice from time to time. It’s curious because if microservices are so much simpler amd easier to maintain than their monolithic counterparts, and allow engineers to be more productive, then why don’t they simply take over like weeds? Are many of my colleagues and I just shitty engineers or is there something deeper going on? At some companies and environments microservices do proliferate and in others they don’t. This is especially curious because people often follow the path of least resistance – they want to get their job done and move on.

In my experience, what I’ve found is that people tend to microservices only when the process of spinning up a new microservice is easy. If that’s the case, then it’s easier to create a new microservice for a new piece of functionality. Otherwise, it’s easier and faster and cheaper to simply continue tacking on new functionality to an existing monolithic application until it becomes zombieware, at which point you still add on new code because nobody wants to deal with that undead elephant of a problem.

As an extremely astute colleague and friend of mine, Henry Seurer, once said: “The app is the deployment pipeline.” By saying that, Henry was basically saying that engineers just leverage the existing app to create new functionality because it’s the easiest thing to do. Thus, in order for organizations to have a thriving microservice ecosystem, they must enable engineers to easily provision hosts and deploy services/containers automatically using easy-to-create pipelines.

So I had my first skip-one meeting today (i.e. my first one-on-one meeting with my boss’ boss) for my current team. It was interesting. My director is a sharp guy – I like him, and I feel like he’s trustworthy. One of the things that I found interesting was how he tried to get to know me better by probing me on fundamentals. One question in particular that I found interesting was: Why should we continue deploying our web services using Docker containers? It was the first time in quite some time that I thought about this question, and I’ll list the two main reasons that I gave my director. They are the reasons that I think are the most critical for large scale enterprise environments.

1. Containers are a lighter weight form of virtualization. Virtualization allows one to run multiple services on a single host. This is useful because if your host isn’t precisely calibrated to fit your service, your hardware resources become idle. For example, an idle CPU, unused RAM, empty disk space, and an idle NIC and all consequences of undersized applications running on commodity hardware that isn’t precisely chosen for the application it runs. And of course, in our world of data centers and clouds, we don’t want to custom build hosts, we want them to be commoditized for optimal economics. By using virtualization, you can have multiple services that make better use of your hardware’s resources. But, virtualization isn’t perfect. Virtualization would be perfect if it didn’t come with it’s own overhead. Virtualization, itself, requires additional CPU cycles, more MBs used by RAM, more HDD pages, etc. This cost can be considerable because you’re basically emulating an entire host and operating system; this cost can be enormous when you extrapolate it to an entire data center or even larger regions and WANs. The diagram below from a nice ZDNet article on the subject shows how Docker is a lighter weight form of VMs in some detail. Containers reduce the cost of the overhead that comes with traditional forms of virtualization.
2. Containers let you deploy your application with greater automation and reliability. So the aforementioned ZDNet article discusses how containers lend themselves more to CICD. The reason they do so is because it’s easier to deploy your service on not just commoditized hardware, but to leverage PaaS more. For example, suppose you have a Java 8 RESTful web service running on Tomcat 7. You don’t need a complicated automated deployment pipeline that deploys Java 8, Tomcat 7, and the WAR file (or worst yet get an ops team to manually deploy Tomcat 7 and Java 8 and only use a CICD pipeline to deploy your WAR file). Instead, you can just deploy a single container that contains all of Java 8, Tomcat 7, and your web service’s WAR file as a single deployment artifact. Since you’re just deploying a single container, everything going out as the product of what the pipeline pushes, you can incorporate tests to ensure everything works as a whole, leaving less of a need than ever before for shitty manual deployment processes.

Anyways, that’s it, I hope that these two core reasons benefit you. They sure helped me. 🙂

So I’ve recently started re-reading a book called “The Well Grounded Java Developer”, that I started reading a couple of years ago. Granted, Java 7 isn’t the latest and greatest Java version, but IMO, I would do well to learn Java 7 in detail, as well as complementary JVM languages like Groovy, Clojure, and Scala. Of course, I would also do well to learn Java 8 and 9 in depth as well. One the early sections in the Well Grounded Java Developer – Sec. 1.3, The Changes in Project Coin, describes the new Final Rethrow capability of the Java compiler. When I first read about this functionality, I was confused about it, so I’m writing a brief post to succinctly explain what it is and its effect on method signatures in Java 7.

In Java 6, methods signatures were required to throw the types of all caught exceptions (that aren’t runtime exceptions) as defined by the catch blocks. For example, consider the following code:

[code language=”java”]
public void testTypicalRethrow() throws Exception {
try {
throwSqlException();
throwIoException();
} catch (Exception e) {
throw e;
}
}

private void throwSqlException() throws SQLException {
throw new SQLException();
}

private void throwIoException() throws IOException {
throw new IOException();
}
[/code]

In the above code, we want a generic catch block (because we want to just rethrow the SQLException and the IOException). We also want a precise method signature (i.e. testTypicalRethrow() throws IOException, SQLException), but Java 6 forces us to throw an Exception, not either an IOException or a SQLException. So, in Java 6, the following line (in the context of the above code) would result in a compiler error.

public void testTypicalRethrow() throws IOException, SQLException {

However, the Java 7 compiler is smarter. The above line is now legal. Now, the following code is legal:
[code language=”java”]
public void testTypicalRethrow() throws IOException, SQLException {
try {
throwSqlException();
throwIoException();
} catch (Exception e) {
throw e;
}
}
[/code]
Previously, this signature of testTypicalRethrow would be illegal, but because the increased compiler intelligence brought on by the Final Rethrow feature allows this code to now be compiled, our method signatures can be more precise, while also having generic catch blocks.

The Final Rethrow capability allows the method signature to throw all possible types of exceptions that are actually caught, without relying of the catch blocks, themselves.

One last point… Ironically, the final keyword isn’t actually required, as you can see from the code above. So long as the original caught exception is rethrown, the compiler is intelligent enough to allow for the more restrictive method signature.

Happy coding.

TL;DR

Use horizontal scaling if you can, it’s usually preferable to vertical scaling. However, if you need run large monolithic apps, then vertical scaling is probably your best bet, although you should try to cut back on app size and resource utilization ASAP.

In slightly more detail, I would propose the following pros-and-cons list of horizontal vs. vertical scaling:

Dimension of Comparison	Horizontal Scaling	Vertical Scaling
Cost	less expensive (through commodity hardware components)	more expensive (specialized servers)
Availability of IT Resources	IT resources instantly available	IT resources normally instantly available
Replication and Backup	nodes that can often serve as backup for one another	other means will need to be found to replicate and backup data
Scaling Limitations	often automated and not limited by hardware capacity	often manual and limited by maximum hardware capacity
Failover	usually builtin with nodes that can easily substitute for other nodes	often a single point of failure
Ability to Run Large Applications	App size and utilization is usually limited by smaller size hosts	can often run large monolithic applications

My justification for the proposal of the above pros-and-cons list is the long explanation below, not the short abstract above.

So I’ve just started reading a book called “Cloud Computing: Concepts, Technology & Architecture” (CCCTA). So far the book is quite good. It got mixed reviews on Amazon, but from what I’m reading so far, it contains useful basic info on cloud theory. One of CCCTA’s sections describes pros and cons for horizontal scaling vs. vertical scaling. Frankly, I largely disagree with the author’s pros-and-cons list. The pros and cons of horizontal and vertical scaling can be found on page 38, in a table (Table 3.1). The table contains the following data:

Horizontal Scaling	Vertical Scaling
less expensive (through commodity hardware components)	more expensive (specialized servers)
IT resources instantly available	IT resources normally instantly available
resource replication and automated scaling	additional setup is normally needed
additional IT resources needed	no additional IT resources needed
not limited by hardware capacity	limited by maximum hardware capacity

 

Some points about this table:

1. I agree with the first row. Horizontal scaling scaling is generally cheaper than vertical scaling.
2. I also agree with the second row. Vertical scaling can sometimes require more specialized hosts and consequently more specialized IT resources that may not be immediately available.
3. Row three is where I start to disagree with the author. True, horizontal scaling does often come with resource replication and automated scaling. But the statement “additional setup is normally needed” for vertical scaling (while probably true) is largely moot. What really matters are the IT resources which may not be immediately available (see row 2). Furthermore, this row misses the point that while resource replication and automated scaling are often benefits of horizontal scaling, they aren’t benefits of vertical scaling. Replication and backups still need to be thought of separately, because you don’t have inbuilt backups in your cluster/group of nodes. Furthermore, scaling is manual, not automatic, since you always need to provision larger hosts.
4. The fourth row, is IMO, incorrect. We need additional IT resources for horizontal scaling but not for vertical scaling… Huh? Don’t we need additional IT resources to setup and provision and switchover to the new larger host? Seems like the amount of labor involved for vertical scaling is at least as much as horizontal scaling.
5. As for the fifth row of the table, I agree with it. The limits of scale for horizontal scaling is, in my experience, much larger than the scaling limits of vertical scaling.

If this table was correct and complete, I don’t think that anybody would ever choose vertical scaling, because based solely on the above table, vertical scaling has no advantages over horizontal scaling. But, people do choose vertical scaling, and for good reason. The reason vertical scaling is sometimes chosen is because for all the benefits that horizontal scaling offers (and IMO Table 3.1 of CCCTA is largely correct about those benefits), not all the apps we have to deal with scale out horizontally. For big, monolithic, resource hogging apps, sometimes vertical scaling is the best choice. The high business value of some of these applications can dramatically outweigh the cost of the specialized hardware. Sure, ideally, we wouldn’t want behemoth apps that are a pain to maintain, but sometimes it’s just the best course of action. For example, mission critical databases are often such apps that demand vertical scaling. Sure, you could switch those DBs over to a horizontally scalable NoSQL solution, but oftentimes you can’t. When your Oracle DB contains real-time critical data related to bookings, and a plethora of apps access that DB directly, and you need to scale up, vertically scaling that DB might just be the fastest and cheapest thing to do (when you factor in the cost of changing all applications that connect to that DB). That being said, IMO horizontal scaling is the best longterm strategy, whereas vertical scaling might be what you need in the short-term.

Outside of table, this section of CCCTA also contains another “con” of vertical scaling that I really disagree with. On page 37, it reads:

Vertical scaling is less common in cloud environments due to the downtime required while the replacement is taking place.

IMO, this statement is wrong for two reasons:

1. Vertical scaling doesn’t necessarily need to require any downtime. You put your old smaller host behind an LB, then setup the new larger host, then switch traffic over to the new host at the LB level. Poof… near-zero downtime. Of course, there are exceptions to this rule, but in my experience, they are few and far between.
2. Vertical scaling is less common in cloud environments not because of downtime, but because of cost and resiliency. Bigger hosts cost more money. Like, a lot more money. And, with vertical scaling, you still have a single point of failure. Moreover, you can only scale up vertically so far, before host costs become either prohibitive, or simply unavailable.

With all that said, CCCTA is still IMO a decent book so far. I just take issue with this individual section. For the reasons described above in my “long explanation”, I would revise Table 3.1 in CCCTA to be my own table of reasons given at the start of this post.