Monthly Archives: July 2016

Go Programming Language: An Introductory Tutorial


What’s the Go Programming Language?

Go is a recent language which sits neatly in the middle of the landscape, providing lots of good features and deliberately omitting many bad ones. It compiles fast, runs fast-ish, includes a runtime and garbage collection, has a simple static type system and dynamic interfaces, and an excellent standard library.

Go and OOP

OOP is one of those features that Go deliberately omits. It has no subclassing, and so there are no inheritance diamonds or super calls or virtual methods to trip you up. Still, many of the useful parts of OOP are available in other ways.

*Mixins* are available by embedding structs anonymously, allowing their methods to be called directly on the containing struct (see embedding). Promoting methods in this way is called *forwarding*, and it’s not the same as subclassing: the method will still be invoked on the inner, embedded struct.

Embedding also doesn’t imply polymorphism. While `A` may have a `B`, that doesn’t mean it is a `B` — functions which take a `B` won’t take an `A` instead. For that, we need interfaces, which we’ll encounter briefly later.

Meanwhile, Go takes a strong position on features that can lead to confusion and bugs. It omits OOP idioms such as inheritance and polymorphism, in favor of composition and simple interfaces. It downplays exception handling in favour of explicit errors in return values. There is exactly one correct way to lay out Go code, enforced by the gofmt tool. And so on.

Go is also a great language for writing concurrent programs: programs with many independently running parts. An obvious example is a webserver: Every request runs separately, but requests often need to share resources such as sessions, caches, or notification queues. This means skilled Go programmers need to deal with concurrent access to those resources.

While the Go language has an excellent set of low-level features for handling concurrency, using them directly can become complicated. In many cases, a handful of reusable abstractions over those low-level mechanisms makes life much easier.

In today’s Go programming tutorial, we’re going to look at one such abstraction: A wrapper which can turn any data structure into a transactional service. We’ll use a Fund type as an example – a simple store for our startup’s remaining funding, where we can check the balance and make withdrawals.

In this introduction to programming in Go, we’ll build the service in small steps, making a mess along the way and then cleaning it up again. Along the way, we’ll encounter lots of cool Go features, including:

  • Struct types and methods
  • Unit tests and benchmarks
  • Goroutines and channels
  • Interfaces and dynamic typing

A Simple Fund

Let’s write some code to track our startup’s funding. The fund starts with a given balance, and money can only be withdrawn (we’ll figure out revenue later).

This graphic depicts a simple goroutine example using the Go programming language.

Go is deliberately not an object-oriented language: There are no classes, objects, or inheritance. Instead, we’ll declare a struct type called Fund, with a simple function to create new fund structs, and two public methods.

fund.go

package funding

type Fund struct {
    // balance is unexported (private), because it's lowercase
    balance int
}

// A regular function returning a pointer to a fund
func NewFund(initialBalance int) *Fund {
    // We can return a pointer to a new struct without worrying about
    // whether it's on the stack or heap: Go figures that out for us.
    return &Fund{
        balance: initialBalance,
    }
}

// Methods start with a *receiver*, in this case a Fund pointer
func (f *Fund) Balance() int {
    return f.balance
}

func (f *Fund) Withdraw(amount int) {
    f.balance -= amount
}

Testing with benchmarks

Next we need a way to test Fund. Rather than writing a separate program, we’ll use Go’s testing package, which provides a framework for both unit tests and benchmarks. The simple logic in our Fund isn’t really worth writing unit tests for, but since we’ll be talking a lot about concurrent access to the fund later on, writing a benchmark makes sense.

Benchmarks are like unit tests, but include a loop which runs the same code many times (in our case, fund.Withdraw(1)). This allows the framework to time how long each iteration takes, averaging out transient differences from disk seeks, cache misses, process scheduling, and other unpredictable factors.

The testing framework wants each benchmark to run for at least 1 second (by default). To ensure this, it will call the benchmark multiple times, passing in an increasing “number of iterations” value each time (the b.Nfield), until the run takes at least a second.

For now, our benchmark will just deposit some money and then withdraw it one dollar at a time.

fund_test.go

package funding

import "testing"

func BenchmarkFund(b *testing.B) {
    // Add as many dollars as we have iterations this run
    fund := NewFund(b.N)

    // Burn through them one at a time until they are all gone
    for i := 0; i < b.N; i++ {
        fund.Withdraw(1)
    }

    if fund.Balance() != 0 {
        b.Error("Balance wasn't zero:", fund.Balance())
    }
}

Now let’s run it:

$ go test -bench . funding
testing: warning: no tests to run
PASS
BenchmarkWithdrawals    2000000000             1.69 ns/op
ok      funding    3.576s

That went well. We ran two billion (!) iterations, and the final check on the balance was correct. We can ignore the “no tests to run” warning, which refers to the unit tests we didn’t write (in later Go programming examples in this tutorial, the warning is snipped out).

Concurrent Access

Now let’s make the benchmark concurrent, to model different users making withdrawals at the same time. To do that, we’ll spawn ten goroutines and have each of them withdraw one tenth of the money.

How would we structure muiltiple concurrent goroutines in the Go language?

Goroutines are the basic building block for concurrency in the Go language. They are green threads – lightweight threads managed by the Go runtime, not by the operating system. This means you can run thousands (or millions) of them without any significant overhead. Goroutines are spawned with the gokeyword, and always start with a function (or method call):

// Returns immediately, without waiting for `DoSomething()` to complete
go DoSomething()

Often, we want to spawn off a short one-time function with just a few lines of code. In this case we can use a closure instead of a function name:

go func() {
    // ... do stuff ...
}() // Must be a function *call*, so remember the ()

Once all our goroutines are spawned, we need a way to wait for them to finish. We could build one ourselves using channels, but we haven’t encountered those yet, so that would be skipping ahead.

For now, we can just use the WaitGroup type in Go’s standard library, which exists for this very purpose. We’ll create one (called “wg”) and call wg.Add(1) before spawning each worker, to keep track of how many there are. Then the workers will report back using wg.Done(). Meanwhile in the main goroutine, we can just say wg.Wait() to block until every worker has finished.

Inside the worker goroutines in our next example, we’ll use defer to call wg.Done().

defer takes a function (or method) call and runs it immediately before the current function returns, after everything else is done. This is handy for cleanup:

func() {
    resource.Lock()
    defer resource.Unlock()

    // Do stuff with resource
}()

This way we can easily match the Unlock with its Lock, for readability. More importantly, a deferred function will run even if there is a panic in the main function (something that we might handle via try-finally in other languages).

Lastly, deferred functions will execute in the reverse order to which they were called, meaning we can do nested cleanup nicely (similar to the C idiom of nested gotos and labels, but much neater):

func() {
    db.Connect()
    defer db.Disconnect()

    // If Begin panics, only db.Disconnect() will execute
    transaction.Begin()
    defer transaction.Close()

    // From here on, transaction.Close() will run first,
    // and then db.Disconnect()

    // ...
}()

OK, so with all that said, here’s the new version:

fund_test.go

package funding

import (
    "sync"
    "testing"
)

const WORKERS = 10

func BenchmarkWithdrawals(b *testing.B) {
    // Skip N = 1
    if b.N < WORKERS {
        return
    }

    // Add as many dollars as we have iterations this run
    fund := NewFund(b.N)

    // Casually assume b.N divides cleanly
    dollarsPerFounder := b.N / WORKERS

    // WaitGroup structs don't need to be initialized
    // (their "zero value" is ready to use).
    // So, we just declare one and then use it.
    var wg sync.WaitGroup

    for i := 0; i < WORKERS; i++ {
        // Let the waitgroup know we're adding a goroutine
        wg.Add(1)
        
        // Spawn off a founder worker, as a closure
        go func() {
            // Mark this worker done when the function finishes
            defer wg.Done()

            for i := 0; i < dollarsPerFounder; i++ {
                fund.Withdraw(1)
            }
            
        }() // Remember to call the closure!
    }

    // Wait for all the workers to finish
    wg.Wait()

    if fund.Balance() != 0 {
        b.Error("Balance wasn't zero:", fund.Balance())
    }
}

We can predict what will happen here. The workers will all execute Withdraw on top of each other. Inside it, f.balance -= amount will read the balance, subtract one, and then write it back. But sometimes two or more workers will both read the same balance, and do the same subtraction, and we’ll end up with the wrong total. Right?

$ go test -bench . funding
BenchmarkWithdrawals    2000000000             2.01 ns/op
ok      funding    4.220s

No, it still passes. What happened here?

Remember that goroutines are green threads – they’re managed by the Go runtime, not by the OS. The runtime schedules goroutines across however many OS threads it has available. At the time of writing this Go language tutorial, Go doesn’t try to guess how many OS threads it should use, and if we want more than one, we have to say so. Finally, the current runtime does not preempt goroutines – a goroutine will continue to run until it does something that suggests it’s ready for a break (like interacting with a channel).

All of this means that although our benchmark is now concurrent, it isn’t parallel. Only one of our workers will run at a time, and it will run until it’s done. We can change this by telling Go to use more threads, via the GOMAXPROCS environment variable.

$ GOMAXPROCS=4 go test -bench . funding
BenchmarkWithdrawals-4    --- FAIL: BenchmarkWithdrawals-4
    account_test.go:39: Balance wasn't zero: 4238
ok      funding    0.007s

That’s better. Now we’re obviously losing some of our withdrawals, as we expected.

In this Go programming example, the outcome of multiple parallel goroutines is not favorable.

Make it a server

At this point we have various options. We could add an explicit mutex or read-write lock around the fund. We could use a compare-and-swap with a version number. We could go all out and use a CRDT scheme (perhaps replacing the balance field with lists of transactions for each client, and calculating the balance from those).

But we won’t do any of those things now, because they’re messy or scary or both. Instead, we’ll decide that a fund should be a server. What’s a server? It’s something you talk to. In Go, things talk via channels.

Channels are the basic communication mechanism between goroutines. Values are sent to the channel (with channel <- value), and can be received on the other side (with value = <- channel). Channels are “goroutine safe”, meaning that any number of goroutines can send to and receive from them at the same time.

Buffering

Buffering communication channels can be a performance optimization in certain circumstances, but it should be used with great care (and benchmarking!).

However, there are uses for buffered channels which aren’t directly about communication.

For instance, a common throttling idiom creates a channel with (for example) buffer size `10` and then sends ten tokens into it immediately. Any number of worker goroutines are then spawned, and each receives a token from the channel before starting work, and sends it back afterward. Then, however many workers there are, only ten will ever be working at the same time.

By default, Go channels are unbuffered. This means that sending a value to a channel will block until another goroutine is ready to receive it immediately. Go also supports fixed buffer sizes for channels (using make(chan someType, bufferSize)). However, for normal use, this is usually a bad idea.

Imagine a webserver for our fund, where each request makes a withdrawal. When things are very busy, the FundServer won’t be able to keep up, and requests trying to send to its command channel will start to block and wait. At that point we can enforce a maximum request count in the server, and return a sensible error code (like a 503 Service Unavailable) to clients over that limit. This is the best behavior possible when the server is overloaded.

Adding buffering to our channels would make this behavior less deterministic. We could easily end up with long queues of unprocessed commands based on information the client saw much earlier (and perhaps for requests which had since timed out upstream). The same applies in many other situations, like applying backpressure over TCP when the receiver can’t keep up with the sender.

In any case, for our Go example, we’ll stick with the default unbuffered behavior.

We’ll use a channel to send commands to our FundServer. Every benchmark worker will send commands to the channel, but only the server will receive them.

We could turn our Fund type into a server implementation directly, but that would be messy – we’d be mixing concurrency handling and business logic. Instead, we’ll leave the Fund type exactly as it is, and make FundServer a separate wrapper around it.

Like any server, the wrapper will have a main loop in which it waits for commands, and responds to each in turn. There’s one more detail we need to address here: The type of the commands.

A diagram of the fund being used as the server in this Go programming tutorial.

Pointers

We could have made our commands channel take *pointers* to commands (`chan *TransactionCommand`). Why didn’t we?

Passing pointers between goroutines is risky, because either goroutine might modify it. It’s also often less efficient, because the other goroutine might be running on a different CPU core (meaning more cache invalidation).

Whenever possible, prefer to pass plain values around.

In the next section below, we’ll be sending several different commands, each with its own struct type. We want the server’s Commands channel to accept any of them. In an OOP language we might do this via polymorphism: Have the channel take a superclass, of which the individual command types were subclasses. In Go, we use interfaces instead.

An interface is a set of method signatures. Any type that implements all of those methods can be treated as that interface (without being declared to do so). For our first run, our command structs won’t actually expose any methods, so we’re going to use the empty interface, interface{}. Since it has no requirements, any value (including primitive values like integers) satisfies the empty interface. This isn’t ideal – we only want to accept command structs – but we’ll come back to it later.

For now, let’s get started with the scaffolding for our server:

server.go

package funding

type FundServer struct {
    Commands chan interface{}
    fund Fund
}

func NewFundServer(initialBalance int) *FundServer {
    server := &FundServer{
        // make() creates builtins like channels, maps, and slices
        Commands: make(chan interface{}),
        fund: NewFund(initialBalance),
    }

    // Spawn off the server's main loop immediately
    go server.loop()
    return server
}

func (s *FundServer) loop() {
    // The built-in "range" clause can iterate over channels,
    // amongst other things
    for command := range s.Commands {
    
        // Handle the command
        
    }
}

Now let’s add a couple of struct types for the commands:

type WithdrawCommand struct {
    Amount int
}

type BalanceCommand struct {
    Response chan int
}

The WithdrawCommand just contains the amount to withdraw. There’s no response. The BalanceCommand does have a response, so it includes a channel to send it on. This ensures that responses will always go to the right place, even if our fund later decides to respond out-of-order.

Now we can write the server’s main loop:

func (s *FundServer) loop() {
    for command := range s.Commands {

        // command is just an interface{}, but we can check its real type
        switch command.(type) {

        case WithdrawCommand:
            // And then use a "type assertion" to convert it
            withdrawal := command.(WithdrawCommand)
            s.fund.Withdraw(withdrawal.Amount)

        case BalanceCommand:
            getBalance := command.(BalanceCommand)
            balance := s.fund.Balance()
            getBalance.Response <- balance

        default:
            panic(fmt.Sprintf("Unrecognized command: %v", command))
        }
    }
}

Hmm. That’s sort of ugly. We’re switching on the command type, using type assertions, and possibly crashing. Let’s forge ahead anyway and update the benchmark to use the server.

func BenchmarkWithdrawals(b *testing.B) {
    // ...

    server := NewFundServer(b.N)

    // ...

    // Spawn off the workers
    for i := 0; i < WORKERS; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for i := 0; i < dollarsPerFounder; i++ {
                server.Commands <- WithdrawCommand{ Amount: 1 }
            }
        }()
    }

    // ...

    balanceResponseChan := make(chan int)
    server.Commands <- BalanceCommand{ Response: balanceResponseChan }
    balance := <- balanceResponseChan

    if balance != 0 {
        b.Error("Balance wasn't zero:", balance)
    }
}

That was sort of ugly too, especially when we checked the balance. Never mind. Let’s try it:

$ GOMAXPROCS=4 go test -bench . funding
BenchmarkWithdrawals-4     5000000           465 ns/op
ok      funding    2.822s

Much better, we’re no longer losing withdrawals. But the code is getting hard to read, and there are more serious problems. If we ever issue a BalanceCommand and then forget to read the response, our fund server will block forever trying to send it. Let’s clean things up a bit.

Make it a service

A server is something you talk to. What’s a service? A service is something you talk to with an API. Instead of having client code work with the command channel directly, we’ll make the channel unexported (private) and wrap the available commands up in functions.

type FundServer struct {
    commands chan interface{} // Lowercase name, unexported
    // ...
}

func (s *FundServer) Balance() int {
    responseChan := make(chan int)
    s.commands <- BalanceCommand{ Response: responseChan }
    return <- responseChan
}

func (s *FundServer) Withdraw(amount int) {
    s.commands <- WithdrawCommand{ Amount: amount }
}

Now our benchmark can just say server.Withdraw(1) and balance := server.Balance(), and there’s less chance of accidentally sending it invalid commands or forgetting to read responses.

Here is what using the fund as a service might look like in this sample Go language program.

There’s still a lot of extra boilerplate for the commands, but we’ll come back to that later.

Transactions

Eventually, the money always runs out. Let’s agree that we’ll stop withdrawing when our fund is down to its last ten dollars, and spend that money on a communal pizza to celebrate or commiserate around. Our benchmark will reflect this:

// Spawn off the workers
for i := 0; i < WORKERS; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        for i := 0; i < dollarsPerFounder; i++ {

            // Stop when we're down to pizza money
            if server.Balance() <= 10 {
                break
            }
            server.Withdraw(1)
        }
    }()
}

// ...

balance := server.Balance()
if balance != 10 {
    b.Error("Balance wasn't ten dollars:", balance)
}

This time we really can predict the result.

$ GOMAXPROCS=4 go test -bench . funding
BenchmarkWithdrawals-4    --- FAIL: BenchmarkWithdrawals-4
    fund_test.go:43: Balance wasn't ten dollars: 6
ok      funding    0.009s

We’re back where we started – several workers can read the balance at once, and then all update it. To deal with this we could add some logic in the fund itself, like a minimumBalance property, or add another command called WithdrawIfOverXDollars. These are both terrible ideas. Our agreement is amongst ourselves, not a property of the fund. We should keep it in application logic.

What we really need is transactions, in the same sense as database transactions. Since our service executes only one command at a time, this is super easy. We’ll add a Transact command which contains a callback (a closure). The server will execute that callback inside its own goroutine, passing in the raw Fund. The callback can then safely do whatever it likes with the Fund.

Semaphores and errors

In this next example we’re doing two small things wrong.

First, we’re using a `Done` channel as a semaphore to let calling code know when its transaction has finished. That’s fine, but why is the channel type `bool`? We’ll only ever send `true` into it to mean “done” (what would sending `false` even mean?). What we really want is a single-state value (a value that has no value?). In Go, we can do this using the empty struct type: `struct{}`. This also has the advantage of using less memory. In the example we’ll stick with `bool` so as not to look too scary.

Second, our transaction callback isn’t returning anything. As we’ll see in a moment, we can get values out of the callback into calling code using scope tricks. However, transactions in a real system would presumably fail sometimes, so the Go convention would be to have the transaction return an `error` (and then check whether it was `nil` in calling code).

We’re not doing that either for now, since we don’t have any errors to generate.

// Typedef the callback for readability
type Transactor func(fund *Fund)

// Add a new command type with a callback and a semaphore channel
type TransactionCommand struct {
    Transactor Transactor
    Done chan bool
}

// ...

// Wrap it up neatly in an API method, like the other commands
func (s *FundServer) Transact(transactor Transactor) {
    command := TransactionCommand{
        Transactor: transactor,
        Done: make(chan bool),
    }
    s.commands <- command
    <- command.Done
}

// ...

func (s *FundServer) loop() {
    for command := range s.commands {
        switch command.(type) {
        // ...

        case TransactionCommand:
            transaction := command.(TransactionCommand)
            transaction.Transactor(s.fund)
            transaction.Done <- true

        // ...
        }
    }
}

Our transaction callbacks don’t directly return anything, but the Go language makes it easy to get values out of a closure directly, so we’ll do that in the benchmark to set the pizzaTime flag when money runs low:

pizzaTime := false
for i := 0; i < dollarsPerFounder; i++ {

    server.Transact(func(fund *Fund) {
        if fund.Balance() <= 10 {
            // Set it in the outside scope
            pizzaTime = true
            return
        }
        fund.Withdraw(1)
    })

    if pizzaTime {
        break
    }
}

And check that it works:

$ GOMAXPROCS=4 go test -bench . funding
BenchmarkWithdrawals-4     5000000           775 ns/op
ok      funding    4.637s

Nothing But transactions

You may have spotted an opportunity to clean things up some more now. Since we have a generic Transactcommand, we don’t need WithdrawCommand or BalanceCommand anymore. We’ll rewrite them in terms of transactions:

func (s *FundServer) Balance() int {
    var balance int
    s.Transact(func(f *Fund) {
        balance = f.Balance()
    })
    return balance
}

func (s *FundServer) Withdraw(amount int) {
    s.Transact(func (f *Fund) {
        f.Withdraw(amount)
    })
}

Now the only command the server takes is TransactionCommand, so we can remove the whole interface{}mess in its implementation, and have it accept only transaction commands:

type FundServer struct {
    commands chan TransactionCommand
    fund *Fund
}

func (s *FundServer) loop() {
    for transaction := range s.commands {
        // Now we don't need any type-switch mess
        transaction.Transactor(s.fund)
        transaction.Done <- true
    }
}

Much better.

There’s a final step we could take here. Apart from its convenience functions for Balance and Withdraw, the service implementation is no longer tied to Fund. Instead of managing a Fund, it could manage an interface{} and be used to wrap anything. However, each transaction callback would then have to convert the interface{} back to a real value:

type Transactor func(interface{})

server.Transact(func(managedValue interface{}) {
    fund := managedValue.(*Fund)
    // Do stuff with fund ...
})

This is ugly and error-prone. What we really want is compile-time generics, so we can “template” out a server for a particular type (like *Fund).

Unfortunately, Go doesn’t support generics – yet. It’s expected to arrive eventually, once someone figures out some sensible syntax and semantics for it. In the meantime, careful interface design often removes the need for generics, and when they don’t we can get by with type assertions (which are checked at runtime).

So we’re done, right?

Yes.

Well, okay, no.

For instance:

  • A panic in a transaction will kill the whole service.
  • There are no timeouts. A transaction that never returns will block the service forever.
  • If our Fund grows some new fields and a transaction crashes halfway through updating them, we’ll have inconsistent state.
  • Transactions are able to leak the managed Fund object, which isn’t good.
  • There’s no reasonable way to do transactions across multiple funds (like withdrawing from one and depositing in another). We can’t just nest our transactions because it would allow deadlocks.
  • Running a transaction asynchronously now requires a new goroutine and a lot of messing around. Relatedly, we probably want to be able to read the most recent Fund state from elsewhere while a long-running transaction is in progress.

In the next Go programming tutorial, we’ll look at some ways to address these issues.

This article was written by Brendon Hogger, a Toptal Python developer.

Introduction to Kotlin: Android Programming For Humans


In a perfect Android world, the main language of Java is really modern, clear, and elegant. You can write less by doing more, and whenever a new feature appears, developers can use it just by increasing version in Gradle. Then while creating a very nice app, it appears fully testable, extensible, and maintainable. Our activities are not too large and complicated, we can change data sources from database to web without tons of differences, and so on. Sounds great, right? Unfortunately, the Android world isn’t this ideal. Google is still striving for perfection, but we all know that ideal worlds don’t exist. Thus, we have to help ourselves in that great journey in the Android world.

Can Kotlin replace Java?

Kotlin is a popular new player in the Android world. But can it ever replace Java?

What Is Kotlin, and Why Should You Use It?

So, the first language. I think that Java isn’t the master of elegance or clarity, and it is neither modern nor expressive (and I’m guessing you agree). The disadvantage is that below Android N, we are still limited to Java 6 (including some small parts of Java 7). Developers can also attach RetroLambda to use lambda expressions in their code, which is very useful while using RxJava. Above Android N, we can use some of Java 8’s new functionalities, but it’s still that old, heavy Java. Very often I hear Android developers say “I wish Android supported a nicer language, like iOS does with Swift”. And what if I told you that you can use a very nice, simple language, with null safety, lambdas, and many other nice new features? Welcome to Kotlin.

What is Kotlin?

Kotlin is a new language (sometimes referred to as Swift for Android), developed by the JetBrains team, and is now in its 1.0.2 version. What makes it useful in Android development is that it compiles to JVM bytecode, and can be also compiled to JavaScript. It is fully compatible with Java, and Kotlin code can be simply converted to Java code and vice versa (there is a plugin from JetBrains). That means Kotlin can use any framework, library etc. written in Java. On Android, it integrates by Gradle. If you have an existing Android app and you want to implement a new feature in Kotlin without rewriting the whole app, just start writing in Kotlin, and it will work.

But what are the ‘great new features’? Let me list a few:

Optional and Named Function Parameters

fun createDate(day: Int, month: Int, year: Int, hour: Int = 0, minute: Int = 0, second: Int = 0) {
   print("TEST", "$day-$month-$year $hour:$minute:$second")
}

We Can Call Method createDate in Different Ways

createDate(1,2,2016) prints:  ‘1-2-2016 0:0:0’
createDate(1,2,2016, 12) prints: ‘1-2-2016 12:0:0’
createDate(1,2,2016, minute = 30) prints: ‘1-2-2016 0:30:0’

Null Safety

If a variable can be null, code will not compile unless we force them to make it. The following code will have an error – nullableVar may be null:

var nullableVar: String? = “”;
nullableVar.length;

To compile, we have to check if it’s not null:

if(nullableVar){
	nullableVar.length
}

Or, shorter:

nullableVar?.length

This way, if nullableVar is null, nothing happens. Otherwise, we can mark variable as not nullable, without a question mark after type:

var nonNullableVar: String = “”;
nonNullableVar.length;

This code compiles, and if we want to assign null to nonNullableVar, compiler will show an error.

There is also very useful Elvis operator:

var stringLength = nullableVar?.length ?: 0 

Then, when nullableVar is null (so nullableVar?.length returns null), stringLength will have value 0.

Mutable and Immutable Variables

In the example above, I use var when defining a variable. This is mutable, we can reassign it whenever we want. If we want that variable to be immutable (in many cases we should), we use val (as value, not variable):

val immutable: Int = 1

After that, the compiler will not allow us to reassign to immutable.

Lambdas

We all know what a lambda is, so here I will just show how we can use it in Kotlin:

button.setOnClickListener({ view -> Log.d("Kotlin","Click")})

Or if the function is the only or last argument:

button.setOnClickListener { Log.d("Kotlin","Click")}

Extensions

Extensions are a very helpful language feature, thanks to which we can “extend” existing classes, even when they are final or we don’t have access to their source code.

For example, to get a string value from edit text, instead of writing every time editText.text.toString() we can write the function:

fun EditText.textValue(): String{
   return text.toString()
}

Or shorter:

fun EditText.textValue() = text.toString()

And now, with every instance of EditText:

editText.textValue()

Or, we can add a property returning the same:

var EditText.textValue: String
   get() = text.toString()
   set(v) {setText(v)}

Operator Overloading

Sometimes useful if we want to add, multiply, or compare objects. Kotlin allows overloading of binary operators (plus, minus, plusAssign, range, etc.), array operators ( get, set, get range, set range), and equals and unary operations (+a, -a, etc.)

Data Class

How many lines of code do you need to implement a User class in Java with three properties: copy, equals, hashCode, and toString? In Kaotlin you need only one line:

data class User(val name: String, val surname: String, val age: Int)

This data class provides equals(), hashCode() and copy() methods, and also toString(), which prints User as:

User(name=John, surname=Doe, age=23)

Data classes also provide some other useful functions and properties, which you can see in Kotlin documentation.

Anko Extensions

You use Butterknife or Android extensions, don’t you? What if you don’t need to even use this library, and after declaring views in XML just use it from code by its ID (like with XAML in C#):

<Button
        android:id="@+id/loginBtn"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" />
loginBtn.setOnClickListener{}

Kotlin has very useful Anko extensions, and with this you don’t need to tell your activity what is loginBtn, it knows it just by “importing” xml:

import kotlinx.android.synthetic.main.activity_main.*

There are many other useful things in Anko, including starting activities, showing toasts, and so on. This is not the main goal of Anko – it is designed for easily creating layouts from code. So if you need to create a layout programmatically, this is the best way.

This is only a short view of Kotlin. I recommend reading Antonio Leiva’s blog and his book – Kotlin for Android Developers, and of course the official Kotlin site.

What Is MVP and Why?

A nice, powerful, and clear language is not enough. It’s very easy to write messy apps with every language without good architecture. Android developers (mostly ones who are getting started, but also more advanced ones) often give Activity responsibility for everything around them. Activity (or Fragment, or other view) downloads data, sends to save, presents them, responds to user interactions, edits data, manages all child views . . . and often much more. It’s too much for such unstable objects like Activities or Fragments (it’s enough to rotate the screen and Activity says ‘Goodbye….’).

A very good idea is to isolate responsibilities from views and make them as stupid as possible. Views (Activities, Fragments, custom views, or whatever presents data on screen) should be only responsible for managing their subviews. Views should have presenters, who will communicate with model, and tell them what they should do. This, in short, is the Model-View-Presenter pattern (for me, it should be named Model-Presenter-View to show connections between layers).

MVC vs MVP

“Hey, I know something like that, and it’s called MVC!” – didn’t you think? No, MVP is not the same as MVC. In the MVC pattern, your view can communicate with model. While using MVP, you don’t allow any communication between these two layers – the only way View can communicate with Model is through Presenter. The only thing that View knows about Model can be the data structure. View knows how to, for example, display User, but doesn’t know when. Here’s a simple example:

View knows “I’m Activity, I have two EditTexts and one Button. When somebody clicks the button, I should tell it to my presenter, and pass him EditTexts’ values. And that’s all, I can sleep until next click or presenter tells me what to do.”

Presenter knows that somewhere is a View, and he knows what operations this View can perform. He also knows that when he receives two strings, he should create User from these two strings and send data to model to save, and if save is successful, tell the view ‘Show success info’.

Model just knows where data is, where they should be saved, and what operations should be performed on the data.

Applications written in MVP are easy to test, maintain, and reuse. A pure presenter should know nothing about the Android platform. It should be pure Java (or Kotlin, in our case) class. Thanks to this we can reuse our presenter in other projects. We can also easily write unit tests, testing separately Model, View and Presenter.

MVP pattern leads to better, less complex codebase by keeping user interface and business logic truly separate.

A little digression: MVP should be a part of Uncle Bob’s Clean Architecture to make applications even more flexible and nicely architectured. I’ll try to write about that next time.

Sample App with MVP and Kotlin

That’s enough theory, let’s see some code! Okay, let’s try to create a simple app. The main goal for this app is to create user. First screen will have two EditTexts (Name and Surname) and one Button (Save). After entering name and surname and clicking ‘Save’, the app should show ‘User is saved’ and go to the next screen, where saved name and surname is presented. When name or surname is empty, the app should not save user and show an error indicating what’s wrong.

The first thing after creating Android Studio project is to configure Kotlin. You should install Kotlin plugin, and, after restart, in Tools > Kotlin you can click ‘Configure Kotlin in Project’. IDE will add Kotlin dependencies to Gradle. If you have any existing code, you can easily convert it to Kotlin by (Ctrl+Shift+Alt+K or Code > Convert Java file to Kotlin). If something is wrong and the project does not compile, or Gradle does not see Kotlin, you can check code of the app available on GitHub.

Kotlin not only interoperates well with Java frameworks and libraries, it allows you to continue using most of the same tools that you are already familiar with.

Now that we have a project, let’s start by creating our first view – CreateUserView. This view should have the functionalities mentioned earlier, so we can write an interface for that:

interface CreateUserView : View {
   fun showEmptyNameError() /* show error when name is empty */
   fun showEmptySurnameError() /* show error when surname is empty */
   fun showUserSaved() /* show user saved info */
   fun showUserDetails(user: User) /* show user details */
}

As you can see, Kotlin is similar to Java in declaring functions. All of those are functions that return nothing, and the last have one parameter. This is the difference, parameter type comes after name. The View interface is not from Android – it’s our simple, empty interface:

interface View

Basic Presenter’s interface should have a property of View type, and at least on method (onDestroy for example), where this property will be set to null:

interface Presenter<T : View> {
   var view: T?

   fun onDestroy(){
       view = null
   }
}

Here you can see another Kotlin feature – you can declare properties in interfaces, and also implement methods there.

Our CreateUserView needs to communicate with CreateUserPresenter. The only additional function that this Presenter needs is saveUser with two string arguments:

interface CreateUserPresenter<T : View>: Presenter<T> {
   fun saveUser(name: String, surname: String)
}

We also need Model definition – it’s mentioned earlier data class:

data class User(val name: String, val surname: String)

After declaring all interfaces, we can start implementing.

CreateUserPresenter will be implemented in CreateUserPresenterImpl:

class CreateUserPresenterImpl(override var view: CreateUserView?): CreateUserPresenter<CreateUserView> {

   override fun saveUser(name: String, surname: String) {
   }
}

The first line, with class definition:

CreateUserPresenterImpl(override var view: CreateUserView?)

Is a constructor, we use it for assigning view property, defined in interface.

MainActivity, which is our CreateUserView implementation, needs a reference to CreateUserPresenter:

class MainActivity : AppCompatActivity(), CreateUserView {

   private val presenter: CreateUserPresenter<CreateUserView> by lazy {
       CreateUserPresenterImpl(this)
   }

   override fun onCreate(savedInstanceState: Bundle?) {
       super.onCreate(savedInstanceState)
       setContentView(R.layout.activity_main)

       saveUserBtn.setOnClickListener{
           presenter.saveUser(userName.textValue(), userSurname.textValue()) /*use of textValue() extension, mentioned earlier */


       }
   }

   override fun showEmptyNameError() {
       userName.error = getString(R.string.name_empty_error) /* it's equal to userName.setError() - Kotlin allows us to use property */

   }

   override fun showEmptySurnameError() {
       userSurname.error = getString(R.string.surname_empty_error)
   }

   override fun showUserSaved() {
       toast(R.string.user_saved) /* anko extension - equal to Toast.makeText(this, R.string.user_saved, Toast.LENGTH_LONG) */

   }

   override fun showUserDetails(user: User) {
      
   }

override fun onDestroy() {
   presenter.onDestroy()
}
}

At the beginning of the class, we defined our presenter:

private val presenter: CreateUserPresenter<CreateUserView> by lazy {
       CreateUserPresenterImpl(this)
}

It is defined as immutable (val), and is created by lazy delegate, which will be assigned the first time it is needed. Moreover, we are sure that it will not be null (no question mark after definition).

When the User clicks on the Save button, View sends information to Presenter with EditTexts values. When that happens, User should be saved, so we have to implement saveUser method in Presenter (and some of the Model’s functions):

override fun saveUser(name: String, surname: String) {
   val user = User(name, surname)
   when(UserValidator.validateUser(user)){
       UserError.EMPTY_NAME -> view?.showEmptyNameError()
       UserError.EMPTY_SURNAME -> view?.showEmptySurnameError()
       UserError.NO_ERROR -> {
           UserStore.saveUser(user)
view?.showUserSaved()
           view?.showUserDetails(user)
       }
   }
}

When a User is created, it is sent to UserValidator to check for validity. Then, according to the result of validation, proper method is called. The when() {} construct is same as switch/case in Java. But it is more powerful – Kotlin allows use of not only enum or int in ‘case’, but also ranges, strings or object types. It must contain all possibilities or have an else expression. Here, it covers all UserError values.

By using view?.showEmptyNameError() (with a question mark after view), we are protected from NullPointer. View can be nulled in onDestroy method, and with this construction, nothing will happen.

When a User model has no errors, it tells UserStore to save it, and then instruct View to show success and show details.

As mentioned earlier, we have to implement some model things:

enum class UserError {
   EMPTY_NAME,
   EMPTY_SURNAME,
   NO_ERROR
}

object UserStore {
   fun saveUser(user: User){
       //Save user somewhere: Database, SharedPreferences, send to web...
   }
}

object UserValidator {

   fun validateUser(user: User): UserError {
       with(user){
           if(name.isNullOrEmpty()) return UserError.EMPTY_NAME
           if(surname.isNullOrEmpty()) return UserError.EMPTY_SURNAME
       }

       return UserError.NO_ERROR
   }
}

The most interesting thing here is UserValidator. By using the object word, we can create a singleton class, with no worries about threads, private constructors and so on.

Next thing – in validateUser(user) method, there is with(user) {} expression. Code within such block is executed in context of object, passed in with name and surname are User’s properties.

There is also another little thing. All above code, from enum to UserValidator, definition is placed in one file (definition of User class is also here). Kotlin does not force you to have each public class in single file (or name class exactly as file). Thus, if you have some short pieces of related code (data classes, extensions, functions, constants – Kotlin doesn’t require class for function or constant), you can place it in one single file instead of spreading through all files in project.

When a user is saved, our app should display that. We need another View – it can be any Android view, custom View, Fragment or Activity. I chose Activity.

So, let’s define UserDetailsView interface. It can show user, but it should also show an error when user is not present:

interface UserDetailsView {
   fun showUserDetails(user: User)
   fun showNoUserError()
}

Next, UserDetailsPresenter. It should have a user property:

interface UserDetailsPresenter<T: View>: Presenter<T> {
   var user: User?
}

This interface will be implemented in UserDetailsPresenterImpl. It has to override user property. Every time this property is assigned, user should be refreshed on the view. We can use a property setter for this:

class UserDetailsPresenterImpl(override var view: UserDetailsView?): UserDetailsPresenter<UserDetailsView> {
   override var user: User? = null
       set(value) {
           field = value
           if(field != null){
               view?.showUserDetails(field!!)
           } else {
               view?.showNoUserError()
           }
       }

}

UserDetailsView implementation, UserDetailsActivity, is very simple. Just like before, we have a presenter object created by lazy loading. User to display should be passed via intent. There is one little problem with this for now, and we will solve it in a moment. When we have user from intent, View needs to assign it to their presenter. After that, user will be refreshed on the screen, or, if it is null, the error will appear (and activity will finish – but presenter doesn’t know about that):

class UserDetailsActivity: AppCompatActivity(), UserDetailsView {

   private val presenter: UserDetailsPresenter<UserDetailsView> by lazy {
       UserDetailsPresenterImpl(this)
   }

   override fun onCreate(savedInstanceState: Bundle?) {
       super.onCreate(savedInstanceState)
       setContentView(R.layout.activity_user_details)

       val user = intent.getParcelableExtra<User>(USER_KEY)
       presenter.user = user
   }

   override fun showUserDetails(user: User) {
       userFullName.text = "${user.name} ${user.surname}"
   }

   override fun showNoUserError() {
       toast(R.string.no_user_error)
       finish()
   }

override fun onDestroy() {
   presenter.onDestroy()
}

}

Passing objects via intents requires that this object implements Parcelable interface. This is very ‘dirty’ work. Personally, I hate doing this because of all of the CREATORs, properties, saving, restoring, and so on. Fortunately, there is proper plugin, Parcelable for Kotlin. After installing it, we can generate Parcelable just with one click.

The last thing to do is to implement showUserDetails(user: User) in our MainActivity:

override fun showUserDetails(user: User) {
   startActivity<UserDetailsActivity>(USER_KEY to user) /* anko extension - starts UserDetailsActivity and pass user as USER_KEY in intent */
}

And that’s all.

Demo Android app in Kotlin

We have a simple app that saves a User (actually, it is not saved, but we can add this functionality without touching presenter or view) and presents it on the screen. In the future, if we want to change the way user is presented on the screen, such as from two activities to two fragments in one activity, or two custom views, changes will be only in View classes. Of course, if we don’t change functionality or model’s structure. Presenter, who doesn’t know what exactly View is, won’t need any changes.

What’s Next?

In our app, Presenter is created every time an activity is created. This approach, or its opposite, if Presenter should persist across activity instances, is a subject of much discussion across the Internet. For me, it depends on the app, its needs, and the developer. Sometimes it’s better is to destroy presenter, sometimes not. If you decide to persist one, a very interesting technique is to use LoaderManager for that.

As mentioned before, MVP should be a part of Uncle Bob’s Clean architecture. Moreover, good developers should use Dagger to inject presenters dependencies to activities. It also helps maintain, test, and reuse code in future. Currently, Kotlin works very well with Dagger (before the official release it wasn’t so easy), and also with other helpful Android libraries.

Wrap Up

For me, Kotlin is a great language. It’s modern, clear, and expressive while still being developed by great people. And we can use any new release on any Android device and version. Whatever makes me angry at Java, Kotlin improves.

Of course, as I said nothing is ideal. Kotlin also have some disadvantages. The newest gradle plugin versions (mainly from alpha or beta) don’t work well with this language. Many people complain that build time is a bit longer than pure Java, and apks have some additional MB’s. But Android Studio and Gradle are still improving, and phones have more and more space for apps. That’s why I believe Kotlin can be a very nice language for every Android developer. Just give it a try, and share in the comments section below what you think.

Source code of the sample app is available on Github: github.com/tomaszczura/AndroidMVPKotlin

This article was written by Tomasz Czura, a Toptal Java developer.

Tips And Tools For Optimizing Android Apps


Android devices have a lot of cores, so writing smooth apps is a simple task for anyone, right? Wrong. As everything on Android can be done in a lot of different ways, picking the best option can be tough. If you want to choose the most efficient method, you have to know what’s happening under the hood. Luckily, you don’t have to rely on your feelings or sense of smell, since there’s a lot of tools out there that can help you find bottlenecks by measuring and describing what’s going on. Properly optimized and smooth apps greatly improve the user experience, and also drain less battery.

Let’s see some numbers first to consider how important optimization really is. According to a Nimbledroid post, 86% of users (including me) have uninstalled apps after using them only once due to poor performance. If you’re loading some content, you have less than 11 seconds to show it to the user. Only every third user will give you more time. You might also get a lot of bad reviews on Google Play because of it.

Build Better Apps: Android Performance Patterns

Testing your users’ patience is a shortcut to uninstallation.

The first thing every user notices over and over is the app’s startup time. According to another Nimbledroid post, out of the 100 top apps, 40 start in under 2 seconds, and 70 start in under 3 seconds. So if possible, you should generally display some content as soon as possible and delay the background checks and updates a bit.

Always remember, premature optimization is the root of all evil. You should also not waste too much time with micro optimization. You will see the most benefit of optimizing code that runs often. For example, this includes the onDraw() function, which runs every frame, ideally 60 times per second. Drawing is the slowest operation out there, so try redrawing only what you have to. More about this will come later.

Performance Tips

Enough theory, here is a list of some of the things you should consider if performance matters to you.

1. String vs StringBuilder

Let’s say that you have a String, and for some reason you want to append more Strings to it 10 thousand times. The code could look something like this.

String string = "hello";
for (int i = 0; i < 10000; i++) {
    string += " world";
}

You can see on the Android Studio Monitors how inefficient some String concatenation can be. There’s tons of Garbage Collections (GC) going on.

String vs StringBuilder

This operation takes around 8 seconds on my fairly good device, which has Android 5.1.1. The more efficient way of achieving the same goal is using a StringBuilder, like this.

StringBuilder sb = new StringBuilder("hello");
for (int i = 0; i < 10000; i++) {
    sb.append(" world");
}
String string = sb.toString();

On the same device this happens almost instantly, in less than 5ms. The CPU and Memory visualizations are almost totally flat, so you can imagine how big this improvement is. Notice though, that for achieving this difference, we had to append 10 thousand Strings, which you probably don’t do often. So in case you are adding just a couple Strings once, you will not see any improvement. By the way, if you do:

String string = "hello" + " world";

It gets internally converted to a StringBuilder, so it will work just fine.

You might be wondering, why is concatenating Strings the first way so slow? It is caused by the fact that Strings are immutable, so once they are created, they cannot be changed. Even if you think you are changing the value of a String, you are actually creating a new String with the new value. In an example like:

String myString = "hello";
myString += " world";

What you will get in memory is not 1 String “hello world”, but actually 2 Strings. The String myString will contain “hello world”, as you would expect. However, the original String with the value “hello” is still alive, without any reference to it, waiting to be garbage collected. This is also the reason why you should store passwords in a char array instead of a String. If you store a password as a String, it will stay in the memory in human-readable format until the next GC for an unpredictable length of time. Back to the immutability described above, the String will stay in the memory even if you assign it another value after using it. If you, however, empty the char array after using the password, it will disappear from everywhere.

2. Picking the Correct Data Type

Before you start writing code, you should decide what data types you will use for your collection. For example, should you use a Vector or an ArrayList? Well, it depends on your usecase. If you need a thread-safe collection, which will allow only one thread at once to work with it, you should pick a Vector, as it is synchronized. In other cases you should probably stick to an ArrayList, unless you really have a specific reason to use vectors.

How about the case when you want a collection with unique objects? Well, you should probably pick a Set. They cannot contain duplicates by design, so you will not have to take care of it yourself. There are multiple types of sets, so pick one that fits your use case. For a simple group of unique items, you can use a HashSet. If you want to preserve the order of items in which they were inserted, pick a LinkedHashSet. A TreeSet sorts items automatically, so you will not have to call any sorting methods on it. It should also sort the items efficiently, without you having to think of sorting algorithms.

Data dominates. If you’ve chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming.
— Rob Pike’s 5 Rules of Programming

Sorting integers or strings is pretty straightforward. However, what if you want to sort a class by some property? Let’s say you are writing a list of meals you eat, and store their names and timestamps. How would you sort the meals by timestamp from the lowest to highest? Luckily, it’s pretty simple. It’s enough to implement the Comparable interface in the Meal class and override the compareTo() function. To sort the meals by lowest timestamp to highest, we could write something like this.

@Override
public int compareTo(Object object) {
    Meal meal = (Meal) object;
    if (this.timestamp < meal.getTimestamp()) {
        return -1;
    } else if (this.timestamp > meal.getTimestamp()) {
        return 1;
    }
    return 0;
}

3. Location Updates

There are a lot of apps out there which collect the user’s location. You should use the Google Location Services API for that purpose, which contains a lot of useful functions. There is a separate article about using it, so I will not repeat it.

I’d just like to stress some important points from a performance perspective.

First of all, use only the most precise location as you need. For example, if you are doing some weather forecasting, you don’t need the most accurate location. Getting just a very rough area based on the network is faster, and more battery efficient. You can achieve it by setting the priority to LocationRequest.PRIORITY_LOW_POWER.

You can also use a function of LocationRequest called setSmallestDisplacement(). Setting this in meters will cause your app to not be notified about location change if it was smaller than the given value. For example, if you have a map with nearby restaurants around you, and you set the smallest displacement to 20 meters, the app will not be making requests for checking restaurants if the user is just walking around in a room. The requests would be useless, as there wouldn’t be any new nearby restaurant anyway.

The second rule is requesting location updates only as often as you need them. This is quite self explanatory. If you are really building that weather forecast app, you do not need to request the location every few seconds, as you probably don’t have such precise forecasts (contact me if you do). You can use the setInterval() function for setting the required interval in which the device will be updating your app about the location. If multiple apps keep requesting the user’s location, every app will be notified at every new location update, even if you have a higher setInterval() set. To prevent your app from being notified too often, make sure to always set a fastest update interval with setFastestInterval().

And finally, the third rule is requesting location updates only if you need them. If you are displaying some nearby objects on the map every x seconds and the app goes in background, you do not need to know the new location. There is no reason to update the map if the user cannot see it anyway. Make sure to stop listening for location updates when appropriate, preferably in onPause(). You can then resume the updates in onResume().

4. Network Requests

There is a high chance that your app is using the internet for downloading or uploading data. If it is, you have several reasons to pay attention to handling network requests. One of them is mobile data, which is very limited to a lot of people and you shouldn’t waste it.

The second one is battery. Both WiFi and mobile networks can consume quite a lot of it if they are used too much. Let’s say that you want to download 1 kb. To make a network request, you have to wake up the cellular or WiFi radio, then you can download your data. However, the radio will not fall asleep immediately after the operation. It will stay in a fairly active state for about 20-40 more seconds, depending on your device and carrier.

Network Requests

So, what can you do about it? Batch. To avoid waking up the radio every couple seconds, prefetch things that the user might need in the upcoming minutes. The proper way of batching is highly dynamic depending on your app, but if it is possible, you should download the data the user might need in the next 3-4 minutes. One could also edit the batch parameters based on the user’s internet type, or charging state. For example, if the user is on WiFi while charging, you can prefetch a lot more data than if the user is on mobile internet with low battery. Taking all these variables into account can be a tough thing, which only few people would do. Luckily though, there is GCM Network Manager to the rescue!

GCM Network Manager is a really helpful class with a lot of customizable attributes. You can easily schedule both repeating and one-off tasks. At repeating tasks you can set the lowest, as well as the highest repeat interval. This will allow batching not only your requests, but also requests from other apps. The radio has to be woken up only once per some period, and while it’s up, all apps in the queue download and upload what they are supposed to. This Manager is also aware of the device’s network type and charging state, so you can adjust accordingly. You can find more details and samples in this article, I urge you to check it out. An example task looks like this:

Task task = new OneoffTask.Builder()
    .setService(CustomService.class)
    .setExecutionWindow(0, 30)
    .setTag(LogService.TAG_TASK_ONEOFF_LOG)
    .setUpdateCurrent(false)
    .setRequiredNetwork(Task.NETWORK_STATE_CONNECTED)
    .setRequiresCharging(false)
    .build();

By the way, since Android 3.0, if you do a network request on the main thread, you will get a NetworkOnMainThreadException. That will definitely warn you not to do that again.

5. Reflection

Reflection is the ability of classes and objects to examine their own constructors, fields, methods, and so on. It is used usually for backward compatibility, to check if a given method is available for a particular OS version. If you have to use reflection for that purpose, make sure to cache the response, as using reflection is pretty slow. Some widely used libraries use Reflection too, like Roboguice for dependency injection. That’s the reason why you should prefer Dagger 2. For more details about reflection, you can check a separate post.

6. Autoboxing

Autoboxing and unboxing are processes of converting a primitive type to an Object type, or vice versa. In practice it means converting an int to an Integer. For achieving that, the compiler uses the Integer.valueOf()function internally. Converting is not just slow, Objects also take a lot more memory than their primitive equivalents. Let’s look at some code.

Integer total = 0;
for (int i = 0; i < 1000000; i++) {
    total += i;
}

While this takes 500ms on average, rewriting it to avoid autoboxing will speed it up drastically.

int total = 0;
for (int i = 0; i < 1000000; i++) {
    total += i;
}

This solution runs at around 2ms, which is 25 times faster. If you don’t believe me, test it out. The numbers will be obviously different per device, but it should still be a lot faster. And it’s also a really simple step to optimize.

Okay, you probably don’t create a variable of type Integer like this often. But what about the cases when it is more difficult to avoid? Like in a map, where you have to use Objects, like Map<Integer, Integer>? Look at the solution many people use.

Map<Integer, Integer> myMap = new HashMap<>();
for (int i = 0; i < 100000; i++) {
    myMap.put(i, random.nextInt());
}

Inserting 100k random ints in the map takes around 250ms to run. Now look at the solution with SparseIntArray.

SparseIntArray myArray = new SparseIntArray();
for (int i = 0; i < 100000; i++) {
    myArray.put(i, random.nextInt());
}

This takes a lot less, roughly 50ms. It’s also one of the easier methods for improving performance, as nothing complicated has to be done, and the code also stays readable. While running a clear app with the first solution took 13MB of my memory, using primitive ints took something under 7MB, so just the half of it.

SparseIntArray is just one of the cool collections that can help you avoid autoboxing. A map like Map<Integer, Long> could be replaced by SparseLongArray, as the value of the map is of type Long. If you look at the source code of SparseLongArray, you will see something pretty interesting. Under the hood, it is basically just a pair of arrays. You can also use a SparseBooleanArray similarly.

If you read the source code, you might have noticed a note saying that SparseIntArray can be slower than HashMap. I’ve been experimenting a lot, but for me SparseIntArray was always better both memory and performance wise. I guess it’s still up to you which you choose, experiment with your use cases and see which fits you the most. Definitely have the SparseArrays in your head when using maps.

7. OnDraw

As I’ve said above, when you are optimizing performance, you will probably see the most benefit in optimizing code which runs often. One of the functions running a lot is onDraw(). It may not surprise you that it’s responsible for drawing views on the screen. As the devices usually run at 60 fps, the function is run 60 times per second. Every frame has 16 ms to be fully handled, including its preparation and drawing, so you should really avoid slow functions. Only the main thread can draw on the screen, so you should avoid doing expensive operations on it. If you freeze the main thread for several seconds, you might get the infamous Application Not Responding (ANR) dialog. For resizing images, database work, etc., use a background thread.

If you think your users won’t notice that drop in frame rate, you are wrong!

I’ve seen some people trying to shorten their code, thinking that it will be more efficient that way. That definitely isn’t the way to go, as shorter code totally doesn’t mean faster code. Under no circumstances should you measure the quality of code by the number of lines.

One of the things you should avoid in onDraw() is allocating objects like Paint. Prepare everything in the constructor, so it’s ready when drawing. Even if you have onDraw() optimized, you should call it only as often as you have to. What is better than calling an optimized function? Well, not calling any function at all. In case you want to draw text, there is a pretty neat helper function called drawText(), where you can specify things like the text, coordinates, and the text color.

8. ViewHolders

You probably know this one, but I cannot skip it. The Viewholder design pattern is a way of making scrolling lists smoother. It is a kind of view caching, which can seriously reduce the calls to findViewById() and inflating views by storing them. It can look something like this.

static class ViewHolder {
    TextView title;
    TextView text;

    public ViewHolder(View view) {
        title = (TextView) view.findViewById(R.id.title);
        text = (TextView) view.findViewById(R.id.text);
    }
}

Then, inside the getView() function of your adapter, you can check if you have a useable view. If not, you create one.

ViewHolder viewHolder;
if (convertView == null) {
    convertView = inflater.inflate(R.layout.list_item, viewGroup, false);
    viewHolder = new ViewHolder(convertView);
    convertView.setTag(viewHolder);
} else {
    viewHolder = (ViewHolder) convertView.getTag();
}

viewHolder.title.setText("Hello World");

You can find a lot of useable info about this pattern around the internet. It can also be used in cases when your list view has multiple different types of elements in it, like some section headers.

9. Resizing Images

Chances are, your app will contain some images. In case you are downloading some JPGs from the web, they can have really huge resolutions. However, the devices they will be displayed on will be a lot smaller. Even if you take a photo with the camera of your device, it needs to be downsized before displaying as the photo resolution is a lot bigger than the resolution of the display. Resizing images before displaying them is a crucial thing. If you’d try displaying them in full resolutions, you’d run out of memory pretty quickly. There is a whole lot written about displaying bitmaps efficiently in the Android docs, I will try summing it up.

So you have a bitmap, but you don’t know anything about it. There’s a useful flag of Bitmaps called inJustDecodeBounds at your service, which allows you to find out the bitmap’s resolution. Let’s assume that your bitmap is 1024×768, and the ImageView used for displaying it is just 400×300. You should keep dividing the bitmap’s resolution by 2 until it’s still bigger than the given ImageView. If you do, it will downsample the bitmap by a factor of 2, giving you a bitmap of 512×384. The downsampled bitmap uses 4x less memory, which will help you a lot with avoiding the famous OutOfMemory error.

Now that you know how to do it, you should not do it. … At least, not if your app relies on images heavily, and it’s not just 1-2 images. Definitely avoid stuff like resizing and recycling images manually, use some third party libraries for that. The most popular ones are Picasso by Square, Universal Image Loader, Fresco by Facebook, or my favourite, Glide. There is a huge active community of developers around it, so you can find a lot of helpful people at the issues section on GitHub as well.

10. Strict Mode

Strict Mode is a quite useful developer tool that many people don’t know about. It’s usually used for detecting network requests or disk accesses from the main thread. You can set what issues Strict Mode should look for and what penalty it should trigger. A google sample looks like this:

public void onCreate() {
    if (DEVELOPER_MODE) {
        StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
                .detectDiskReads()
                .detectDiskWrites()
                .detectNetwork()
                .penaltyLog()
                .build());
        StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
                .detectLeakedSqlLiteObjects()
                .detectLeakedClosableObjects()
                .penaltyLog()
                .penaltyDeath()
                .build());
    }
    super.onCreate();
}

If you want to detect every issue Strict Mode can find, you can also use detectAll(). As with many performance tips, you should not blindly try fixing everything Strict Mode reports. Just investigate it, and if you are sure it’s not an issue, leave it alone. Also make sure to use Strict Mode only for debugging, and always have it disabled on production builds.

Debugging Performance: The Pro Way

Let’s now see some tools that can help you find bottlenecks, or at least show that something is wrong.

1. Android Monitor

This is a tool built into Android Studio. By default, you can find the Android Monitor at the bottom left corner, and you can switch between 2 tabs there. Logcat and Monitors. The Monitors section contains 4 different graphs. Network, CPU, GPU, and Memory. They are pretty self explanatory, so I will just quickly go through them. Here is a screenshot of the graphs taken while parsing some JSON as it is downloaded.

Android Monitor

The Network part shows the incoming and outgoing traffic in KB/s. The CPU part displays the CPU usage in percent. The GPU monitor displays how much time it takes to render the frames of a UI window. This is the most detailed monitor out of these 4, so if you want more details about it, read this.

Lastly we have the Memory monitor, which you will probably use the most. By default it shows the current amount of Free and Allocated memory. You can force a Garbage Collection with it too, to test if the amount of used memory drops down. It has a useful feature called Dump Java Heap, which will create a HPROF file which can be opened with the HPROF Viewer and Analyzer. That will enable you to see how many objects you have allocated, how much memory is taken by what, and maybe which objects are causing memory leaks. Learning how to use this analyzer is not the simplest task out there, but it is worth it. The next thing you can do with the Memory Monitor is do some timed Allocation Tracking, which you can start and stop as you wish. It could be useful at many cases, for example when scrolling or rotating the device.

2. GPU Overdraw

This is a simple helper tool, which you can activate in Developer Options once you have enabled developer mode. Select Debug GPU overdraw, “Show overdraw areas”, and your screen will get some weird colors. It’s ok, that’s what is supposed to happen. The colors mean how many times a particular area was overdrawn. True color means that there was no overdraw, this is what you should aim for. Blue means one overdraw, green means two, pink three, red four.

GPU Overdraw

While seeing true color is the best, you will always see some overdraws, especially around texts, navigation drawers, dialogs and more. So don’t try getting rid of it fully. If your app is blueish or greenish, that’s probably fine. However, if you see too much red on some simple screens, you should investigate what’s going on. It might be too many fragments stacked onto each other, if you keep adding them instead of replacing. As I’ve mentioned above, drawing is the slowest part of apps, so there is no sense drawing something if there will be more than 3 layers drawn onto it. Feel free to check out your favourite apps with it. You will see that even apps with over a billion downloads have red areas, so just take it easy when you are trying to optimize.

3. GPU Rendering

This is another tool from the Developer options, called Profile GPU rendering. Upon selecting it, pick “On screen as bars”. You will notice some colored bars appearing on your screen. Since every application has separate bars, weirdly the status bar has its own ones, and in case you have software navigation buttons, they have their own bars too. Anyway, the bars get updated as you interact with the screen.

GPU Rendering

The bars consist of 3-4 colors, and according to the Android docs, their size indeed matters. The smaller, the better. At the bottom you have blue, which represents the time used to create and update the View’s display lists. If this part is too tall, it means that there is a lot of custom view drawing, or a lot of work done in the onDraw() functions. If you have Android 4.0+, you will see a purple bar above the blue one. This represents the time spent transferring resources to the render thread. Then comes the red part, which represents the time spent by Android’s 2D renderer issuing commands to OpenGL to draw and redraw display lists. At the top is the orange bar, which represents the time the CPU is waiting for the GPU to finish its work. If it’s too tall, the app is doing too much work on the GPU.

If you are good enough, there is one more color above the orange. It is a green line representing the 16 ms threshold. As your goal should be running your app at 60 fps, you have 16 ms to draw every frame. If you don’t make it, some frames might be skipped, the app could become jerky, and the user would definitely notice. Pay special attention to animations and scrolling, that’s where the smoothness matters the most. Even though you can detect some skipped frames with this tool, it won’t really help you figuring out where exactly the problem is.

4. Hierarchy Viewer

This is one of my favourite tools out there, as it’s really powerful. You can start it from Android Studio through Tools -> Android -> Android Device Monitor, or it is also in your sdk/tools folder as “monitor”. You can also find a standalone hierarachyviewer executable there, but as it’s deprecated you should open the monitor. However you open the Android Device Monitor, switch to the Hierarchy Viewer perspective. If you don’t see any running apps assigned to your device, there are a couple things you can do to fix it. Also try checking outthis issue thread, there are people with all kinds of issues and all kinds of solutions. Something should work for you too.

With Hierarchy Viewer, you can get a really neat overview of your view hierarchies (obviously). If you see every layout in a separate XML, you might easily spot useless views. However, if you keep combining the layouts, it can easily get confusing. A tool like this makes it simple to spot, for example, some RelativeLayout, which has just 1 child, another RelativeLayout. That makes one of them removable.

Avoid calling requestLayout(), as it causes traversing of the entire view hierarchy, to find out how big each view should be. If there is some conflict with the measurements, the hierarchy might be traversed multiple times, which if happens during some animation, it will definitely make some frames be skipped. If you want to find out more about how Android draws its views, you can read this. Let’s look at one view as seen in Hierarchy Viewer.

The top right corner contains a button for maximizing the preview of the particular view in a standalone window. Under it you can also see the actual preview of the view in the app. The next item is a number, which represents how many children the given view has, including the view itself. If you select a node (preferably the root one) and press “Obtain layout times” (3 colored circles), you will have 3 more values filled, together with colored circles appearing labelled measure, layout, and draw. It might not be shocking that the measure phase represents the time it took to measure the given view. The layout phase is about the rendering time, while the drawing is the actual drawing operation. These values and colors are relative to each other. Green one means that the view renders in the top 50% of all views in the tree. Yellow means rendering in the slower 50% of all views in the tree, red means that the given view is one of the slowest. As these values are relative, there will always be red ones. You simply cannot avoid them.

Under the values you have the class name, such as “TextView”, an internal view ID of the object, and the android:id of the view, which you set in the XML files. I urge you to build a habit of adding IDs to all views, even if you don’t reference them in the code. It will make identifying the views in Hierarchy Viewer really simple, and in case you have automated tests in your project, it will also make targeting the elements a lot faster. That will save some time for you and your colleagues writing them. Adding IDs to elements added in XML files is pretty straightforward. But what about the dynamically added elements? Well, it turns out to be really simple too. Just create an ids.xml file inside your values folder and type in the required fields. It can look like this:

<resources>
    <item name="item_title" type="id"/>
    <item name="item_body" type="id"/>
</resources>

Then in the code, you can use setId(R.id.item_title). It couldn’t be simpler.

There are a couple more things to pay attention to when optimizing UI. You should generally avoid deep hierarchies while preferring shallow, maybe wide ones. Do not use layouts you don’t need. For example, you can probably replace a group of nested LinearLayouts with either a RelativeLayout, or a TableLayout. Feel free to experiment with different layouts, don’t just always use LinearLayout and RelativeLayout. Also try creating some custom views when needed, it can improve the performance significantly if done well. For instance, did you know that Instagram doesn’t use TextViews for displaying comments?

You can find some more info about Hierarchy Viewer on the Android Developers site with descriptions of different panes, using the Pixel Perfect tool, etc. One more thing I would point out is capturing the views in a .psd file, which can be done by the “Capture the window layers” button. Every view will be in a separate layer, so it’s really simple to hide or change it in Photoshop or GIMP. Oh, that’s another reason to add an ID to every view you can. It will make the layers have names that actually make sense.

You will find a lot more debugging tools in Developer options, so I advise you to activate them and see what are they doing. What could possibly go wrong?

The Android developers site contains a set of best practices for performance. They cover a lot of different areas, including memory management, which I haven’t really talked about. I silently ignored it, because handling memory and tracking memory leaks is a whole separate story. Using a third party library for efficiently displaying images will help a lot, but if you still have memory issues, check out Leak canary made by Square, or read this.

Wrapping Up

So, this was the good news. The bad new is, optimizing Android apps is a lot more complicated. There are a lot of ways of doing everything, so you should be familiar with the pros and cons of them. There usually isn’t any silver bullet solution which has only benefits. Only by understanding what’s happening behind the scenes will you be able to pick the solution which is best for you. Just because your favorite developer says that something is good, it doesn’t necessarily mean that it’s the best solution for you. There are a lot more areas to discuss and more profiling tools which are more advanced, so we might get to them next time.

Make sure you learn from the top developers and top companies. You can find a couple hundred engineering blogs at this link. It’s obviously not just Android related stuff, so if you are interested only in Android, you have to filter the particular blog. I would highly recommend the blogs of Facebook and Instagram. Even though the Instagram UI on Android is questionable, their engineering blog has some really cool articles. For me it’s awesome that it’s so easy to see how things are done in companies which are handling hundreds of millions of users daily, so not reading their blogs seems crazy. The world is changing really fast, so if you aren’t constantly trying to improve, learn from others and use new tools, you will be left behind. As Mark Twain said, a person who doesn’t read has no advantage over one who can’t read.

This article was written by Tibor Kaputa, a Toptal Java developer.

Scaling Scala: How to Dockerize Using Kubernetes


Kubernetes is the new kid on the block, promising to help deploy applications into the cloud and scale them more quickly. Today, when developing for a microservices architecture, it’s pretty standard to choose Scala for creating API servers.

Microservices are replacing classic monolithic back-end servers with multiple independent services that communicate among themselves and have their own processes and resources.

If there is a Scala application in your plans and you want to scale it into a cloud, then you are at the right place. In this article I am going to show step-by-step how to take a generic Scala application and implement Kubernetes with Docker to launch multiple instances of the application. The final result will be a single application deployed as multiple instances, and load balanced by Kubernetes.

All of this will be implemented by simply importing the Kubernetes source kit in your Scala application. Please note, the kit hides a lot of complicated details related to installation and configuration, but it is small enough to be readable and easy to understand if you want to analyze what it does. For simplicity, we will deploy everything on your local machine. However, the same configuration is suitable for a real-world cloud deployment of Kubernetes.

Scale Your Scala Application with Kubernetes

Be smart and sleep tight, scale your Docker with Kubernetes.

What is Kubernetes?

Before going into the gory details of the implementation, let’s discuss what Kubernetes is and why it’s important.

You may have already heard of Docker. In a sense, it is a lightweight virtual machine.

Docker gives the advantage of deploying each server in an isolated environment, very similar to a stand-alone virtual machine, without the complexity of managing a full-fledged virtual machine.

For these reasons, it is already one of the more widely used tools for deploying applications in clouds. A Docker image is pretty easy and fast to build and duplicable, much easier than a traditional virtual machine like VMWare, VirtualBox, or XEN.

Kubernetes complements Docker, offering a complete environment for managing dockerized applications. By using Kubernetes, you can easily deploy, configure, orchestrate, manage, and monitor hundreds or even thousands of Docker applications.

Kubernetes is an open source tool developed by Google and has been adopted by many other vendors. Kubernetes is available natively on the Google cloud platform, but other vendors have adopted it for their OpenShift cloud services too. It can be found on Amazon AWS, Microsoft Azure, RedHat OpenShift, and even more cloud technologies. We can say it is well positioned to become a standard for deploying cloud applications.

Prerequisites

Now that we covered the basics, let’s check if you have all the prerequisite software installed. First of all, you need Docker. If you are using either Windows or Mac, you need the Docker Toolbox. If you are using Linux, you need to install the particular package provided by your distribution or simply follow the official directions.

We are going to code in Scala, which is a JVM language. You need, of course, the Java Development Kit and the scala SBT tool installed and available in the global path. If you are already a Scala programmer, chances are you have those tools already installed.

If you are using Windows or Mac, Docker will by default create a virtual machine named default with only 1GB of memory, which can be too small for running Kubernetes. In my experience, I had issues with the default settings. I recommend that you open the VirtualBox GUI, select your virtual machine default, and change the memory to at least to 2048MB.

VirtualBox memory settings

The Application to Clusterize

The instructions in this tutorial can apply to any Scala application or project. For this article to have some “meat” to work on, I chose an example used very often to demonstrate a simple REST microservice in Scala, called Akka HTTP. I recommend you try to apply source kit to the suggested example before attempting to use it on your application. I have tested the kit against the demo application, but I cannot guarantee that there will be no conflicts with your code.

So first, we start by cloning the demo application:

git clone https://github.com/theiterators/akka-http-microservice

Next, test if everything works correctly:

cd akka-http-microservice
sbt run

Then, access to http://localhost:9000/ip/8.8.8.8, and you should see something like in the following image:

Akka HTTP microservice is running

Adding the Source Kit

Now, we can add the source kit with some Git magic:

git remote add ScalaGoodies https://github.com/sciabarra/ScalaGoodies
git fetch --all
git merge ScalaGoodies/kubernetes

With that, you have the demo including the source kit, and you are ready to try. Or you can even copy and paste the code from there into your application.

Once you have merged or copied the files in your projects, you are ready to start.

Starting Kubernetes

Once you have downloaded the kit, we need to download the necessary kubectl binary, by running:

bin/install.sh

This installer is smart enough (hopefully) to download the correct kubectl binary for OSX, Linux, or Windows, depending on your system. Note, the installer worked on the systems I own. Please do report any issues, so that I can fix the kit.

Once you have installed the kubectl binary, you can start the whole Kubernetes in your local Docker. Just run:

bin/start-local-kube.sh

The first time it is run, this command will download the images of the whole Kubernetes stack, and a local registry needed to store your images. It can take some time, so please be patient. Also note, it needs direct accesses to the internet. If you are behind a proxy, it will be a problem as the kit does not support proxies. To solve it, you have to configure the tools like Docker, curl, and so on to use the proxy. It is complicated enough that I recommend getting a temporary unrestricted access.

Assuming you were able to download everything successfully, to check if Kubernetes is running fine, you can type the following command:

bin/kubectl get nodes

The expected answer is:

NAME        STATUS    AGE
127.0.0.1   Ready     2m

Note that age may vary, of course. Also, since starting Kubernetes can take some time, you may have to invoke the command a couple of times before you see the answer. If you do not get errors here, congratulations, you have Kubernetes up and running on your local machine.

Dockerizing Your Scala App

Now that you have Kubernetes up and running, you can deploy your application in it. In the old days, before Docker, you had to deploy an entire server for running your application. With Kubernetes, all you need to do to deploy your application is:

  • Create a Docker image.
  • Push it in a registry from where it can be launched.
  • Launch the instance with Kubernetes, that will take the image from the registry.

Luckily, it is way less complicated that it looks, especially if you are using the SBT build tool like many do.

In the kit, I included two files containing all the necessary definitions to create an image able to run Scala applications, or at least what is needed to run the Akka HTTP demo. I cannot guarantee that it will work with any other Scala applications, but it is a good starting point, and should work for many different configurations. The files to look for building the Docker image are:

docker.sbt
project/docker.sbt

Let’s have a look at what’s in them. The file project/docker.sbt contains the command to import the sbt-docker plugin:

addSbtPlugin("se.marcuslonnberg" % "sbt-docker" % "1.4.0")

This plugin manages the building of the Docker image with SBT for you. The Docker definition is in the docker.sbt file and looks like this:

imageNames in docker := Seq(ImageName("localhost:5000/akkahttp:latest"))

dockerfile in docker := {
  val jarFile: File = sbt.Keys.`package`.in(Compile, packageBin).value
  val classpath = (managedClasspath in Compile).value
  val mainclass = mainClass.in(Compile, packageBin).value.getOrElse(sys.error("Expected exactly one main class"))
  val jarTarget = s"/app/${jarFile.getName}"
  val classpathString = classpath.files.map("/app/" + _.getName)
    .mkString(":") + ":" + jarTarget
  new Dockerfile {
    from("anapsix/alpine-java:8")
    add(classpath.files, "/app/")
    add(jarFile, jarTarget)
    entryPoint("java", "-cp", classpathString, mainclass)
  }
}

To fully understand the meaning of this file, you need to know Docker well enough to understand this definition file. However, we are not going into the details of the Docker definition file, because you do not need to understand it thoroughly to build the image.

The beauty of using SBT for building the Docker image is that
the SBT will take care of collecting all the files for you.

Note the classpath is automatically generated by the following command:

val classpath = (managedClasspath in Compile).value

In general, it is pretty complicated to gather all the JAR files to run an application. Using SBT, the Docker file will be generated with add(classpath.files, "/app/"). This way, SBT collects all the JAR files for you and constructs a Dockerfile to run your application.

The other commands gather the missing pieces to create a Docker image. The image will be built using an existing image APT to run Java programs (anapsix/alpine-java:8, available on the internet in the Docker Hub). Other instructions are adding the other files to run your application. Finally, by specifying an entry point, we can run it. Note also that the name starts with localhost:5000 on purpose, because localhost:5000 is where I installed the registry in the start-kube-local.sh script.

Building the Docker Image with SBT

To build the Docker image, you can ignore all the details of the Dockerfile. You just need to type:

sbt dockerBuildAndPush

The sbt-docker plugin will then build a Docker image for you, downloading from the internet all the necessary pieces, and then it will push to a Docker registry that was started before, together with the Kubernetes application in localhost. So, all you need is to wait a little bit more to have your image cooked and ready.

Note, if you experience problems, the best thing to do is to reset everything to a known state by running the following commands:

bin/stop-kube-local.sh
bin/start-kube-local.sh

Those commands should stop all the containers and restart them correctly to get your registry ready to receive the image built and pushed by sbt.

Starting the Service in Kubernetes

Now that the application is packaged in a container and pushed in a registry, we are ready to use it. Kubernetes uses the command line and configuration files to manage the cluster. Since command lines can become very long, and also be able to replicate the steps, I am using the configurations files here. All the samples in the source kit are in the folder kube.

Our next step is to launch a single instance of the image. A running image is called, in the Kubernetes language, a pod. So let’s create a pod by invoking the following command:

bin/kubectl create -f kube/akkahttp-pod.yml

You can now inspect the situation with the command:

bin/kubectl get pods

You should see:

NAME                   READY     STATUS    RESTARTS   AGE
akkahttp               1/1       Running   0          33s
k8s-etcd-127.0.0.1     1/1       Running   0          7d
k8s-master-127.0.0.1   4/4       Running   0          7d
k8s-proxy-127.0.0.1    1/1       Running   0          7d

Status actually can be different, for example, “ContainerCreating”, it can take a few seconds before it becomes “Running”. Also, you can get another status like “Error” if, for example, you forget to create the image before.

You can also check if your pod is running with the command:

bin/kubectl logs akkahttp

You should see an output ending with something like this:

[DEBUG] [05/30/2016 12:19:53.133] [default-akka.actor.default-dispatcher-5] [akka://default/system/IO-TCP/selectors/$a/0] Successfully bound to /0:0:0:0:0:0:0:0:9000

Now you have the service up and running inside the container. However, the service is not yet reachable. This behavior is part of the design of Kubernetes. Your pod is running, but you have to expose it explicitly. Otherwise, the service is meant to be internal.

Creating a Service

Creating a service and checking the result is a matter of executing:

bin/kubectl create -f kube/akkahttp-service.yaml
bin/kubectl get svc

You should see something like this:

NAME               CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
akkahttp-service   10.0.0.54                  9000/TCP   44s
kubernetes         10.0.0.1     <none>        443/TCP    3m

Note that the port can be different. Kubernetes allocated a port for the service and started it. If you are using Linux, you can directly open the browser and type http://10.0.0.54:9000/ip/8.8.8.8 to see the result. If you are using Windows or Mac with Docker Toolbox, the IP is local to the virtual machine that is running Docker, and unfortunately it is still unreachable.

I want to stress here that this is not a problem of Kubernetes, rather it is a limitation of the Docker Toolbox, which in turn depends on the constraints imposed by virtual machines like VirtualBox, which act like a computer within another computer. To overcome this limitation, we need to create a tunnel. To make things easier, I included another script which opens a tunnel on an arbitrary port to reach any service we deployed. You can type the following command:

bin/forward-kube-local.sh akkahttp-service 9000

Note that the tunnel will not run in the background, you have to keep the terminal window open as long as you need it and close when you do not need the tunnel anymore. While the tunnel is running, you can open: http://localhost:9000/ip/8.8.8.8 and finally see the application running in Kubernetes.

Final Touch: Scale

So far we have “simply” put our application in Kubernetes. While it is an exciting achievement, it does not add too much value to our deployment. We’re saved from the effort of uploading and installing on a server and configuring a proxy server for it.

Where Kubernetes shines is in scaling. You can deploy two, ten, or one hundred instances of our application by only changing the number of replicas in the configuration file. So let’s do it.

We are going to stop the single pod and start a deployment instead. So let’s execute the following commands:

bin/kubectl delete -f kube/akkahttp-pod.yml
bin/kubectl create -f kube/akkahttp-deploy.yaml

Next, check the status. Again, you may try a couple of times because the deployment can take some time to be performed:

NAME                                   READY     STATUS    RESTARTS   AGE
akkahttp-deployment-4229989632-mjp6u   1/1       Running   0          16s
akkahttp-deployment-4229989632-s822x   1/1       Running   0          16s
k8s-etcd-127.0.0.1                     1/1       Running   0          6d
k8s-master-127.0.0.1                   4/4       Running   0          6d
k8s-proxy-127.0.0.1                    1/1       Running   0          6d

Now we have two pods, not one. This is because in the configuration file I provided, there is the value replica: 2, with two different names generated by the system. I am not going into the details of the configuration files, because the scope of the article is simply an introduction for Scala programmers to jump-start into Kubernetes.

Anyhow, there are now two pods active. What is interesting is that the service is the same as before. We configured the service to load balance between all the pods labeled akkahttp. This means we do not have to redeploy the service, but we can replace the single instance with a replicated one.

We can verify this by launching the proxy again (if you are on Windows and you have closed it):

bin/forward-kube-local.sh akkahttp-service 9000

Then, we can try to open two terminal windows and see the logs for each pod. For example, in the first type:

bin/kubectl logs -f akkahttp-deployment-4229989632-mjp6u

And in the second type:

bin/kubectl logs -f akkahttp-deployment-4229989632-s822x

Of course, edit the command line accordingly with the values you have in your system.

Now, try to access the service with two different browsers. You should expect to see the requests to be split between the multiple available servers, like in the following image:

Kubernets in action

Conclusion

Today we barely scratched the surface. Kubernetes offers a lot more possibilities, including automated scaling and restart, incremental deployments, and volumes. Furthermore, the application we used as an example is very simple, stateless with the various instances not needing to know each other. In the real world, distributed applications do need to know each other, and need to change configurations according to the availability of other servers. Indeed, Kubernetes offers a distributed keystore (etcd) to allow different applications to communicate with each other when new instances are deployed. However, this example is purposefully small enough and simplified to help you get going, focusing on the core functionalities. If you follow the tutorial, you should be able to get a working environment for your Scala application on your machine without being confused by a large number of details and getting lost in the complexity.

This article was written by Michele Sciabarra, a Toptal Scala developer.