Getting Started

You can find the Scale Stack source in the Launchpad bzr repository at lp:scalestack or from a tarball at https://launchpad.net/scalestack/+download. Branching directly from bzr on Launchpad will give you the latest version (recommended for developers). If you grab a tarball, unzip/untar the file and follow the README instructions to get started. A simple:

./configure
make
make install

Should do the trick though. If you have any problems find someone on IRC or send an email to the mailing list.

One of the first modules you'll probably want to experiment with is the echo server. This will listen on some set of sockets and echo anything it receives back to the client. To run the server, use the following command:

shell$ scalestack echo::server::tcp.ports=12345 event::service.threads=4 v
NOTICE [event] Loaded
NOTICE [event::libevent] Loaded
NOTICE [network] Loaded
NOTICE [echo::server] Loaded
NOTICE [network::ip] Loaded
NOTICE [network::ip::tcp] Loaded
NOTICE [echo::server::tcp] Loaded
NOTICE [echo::server::tcp] Listening on 0.0.0.0:12345
NOTICE [kernel] Core now running

The first option specifies the TCP ports the echo server should listen on. The second argument specifies how many threads should be started. Each thread can handle an unlimited number of connections (up until process limits), and connections are spread over all threads as they are accepted. This simple echo server could easily handle tens of thousands of network connections fairly efficiently.

With the server running, you can test it out with the 'nc' (netcat) or 'telnet' command:

shell$ nc localhost 12345
this is a test
this is a test

The data is echoed back as expected.

If you would like to see whats happening under the hood, you can also run this with more 'v' arguments to see more informational and debugging messages.

shell$ scalestack echo::server::tcp.ports=12345 event::service.threads=4 vvv
 DEBUG [kernel] Setting option: module=echo::server::tcp option=ports value=12345
  INFO [echo::server::tcp] Constructed
 DEBUG [kernel] Setting option: module=event::service option=threads value=4
  INFO [event::libevent] Constructed
  INFO [kernel] Setting signal handlers
  INFO [echo::server::tcp] Checking pathname: /usr/local/lib/scalestack/libecho_server_tcp_module.so
  INFO [echo::server::tcp] Loading from: /usr/local/lib/scalestack/libecho_server_tcp_module.so
 DEBUG [echo::server::tcp] Dependencies: echo::server,network::ip::tcp
  INFO [echo::server] Constructed
  INFO [echo::server] Checking pathname: /usr/local/lib/scalestack/libecho_server_module.so
  INFO [echo::server] Loading from: /usr/local/lib/scalestack/libecho_server_module.so  
 DEBUG [echo::server] Dependencies: network
  INFO [network] Constructed
  INFO [network] Checking pathname: /usr/local/lib/scalestack/libnetwork_module.so
  INFO [network] Loading from: /usr/local/lib/scalestack/libnetwork_module.so
 DEBUG [network] Dependencies: event,event::service
  INFO [event] Constructed
  INFO [event] Checking pathname: /usr/local/lib/scalestack/libevent_module.so
  INFO [event] Loading from: /usr/local/lib/scalestack/libevent_module.so
 DEBUG [event] Dependencies:
NOTICE [event] Loaded
 DEBUG [event] Version: 0.1
 DEBUG [event] Author: Eric Day
 DEBUG [event] Title: Event Base Class Module
 DEBUG [event] License: BSD
...
NOTICE [echo::server::tcp] Loaded
 DEBUG [echo::server::tcp] Version: 0.1
 DEBUG [echo::server::tcp] Author: Eric Day
 DEBUG [echo::server::tcp] Title: Echo Server TCP Service
 DEBUG [echo::server::tcp] License: BSD
...

This shows the full bootstrapping process for the echo server module. It firsts tries to locate the shared object representing the tcp echo server and finds two dependencies: the echo::server base module and the network::ip::tcp module. Before the echo::server::tcp module continues loading, it first loads the dependencies, which cascades down into a list of other modules (event and core network modules mainly). Once all dependencies are loaded the modules are started and this is where the listening sockets are created. Once all modules are started the server is up and running.

This kind of automatic dependency tracking and loading is just like what the Linux kernel does when you run 'modprobe' for a particular module. I not only loads the specified module but any dependencies so it is successful. Scale Stack is designed to be extremely modular so you can reuse common network and protocol parsing code in your own application. If you are interested in starting to hack on some modules, take a look at scalestack/echo/server/* in the source code to see how modules are written. Also be sure to talk to folks on IRC and the mailing list as the project is still under heavy development at this time.

Scale Stack at the core is a library that can dynamically load other libraries (or modules). The 'scalestack' command line tool is providing for convenience, but you can easily create your own customized binaries or include Scale Stack into existing projects by simply linking the library and making API calls. Take a look at 'bin/scalestack.cc' at how a simple library instance is created with a couple options.

For example, if you wrote a DNS server called 'mydns', you could create a 'mydns' command by copying 'bin/scalestack.cc' and adding in a few more default options to load the 'mydns' related modules. This way those modules would always be loaded when running this new binary, no need to specify them on the command line like we did the echo::server::tcp module above.

The 'scalestack' binary (actually the library) supports a few other options as well. For example, to get a list of options available for module, you can specify 'h' or 'help' on the command line:

shell$ scalestack echo::server::tcp help

scalestack 0.1

Kernel
  v - Increase verbosity level.
  h, help - Display this help message.
  p, path=PATH - Add path to module search list.
                 (Default: /usr/local/lib/scalestack)

echo::server::tcp
  hosts=HOST[:PORT][,HOST[:PORT]]... - List of host/port pairs listen on.
  ports=PORT[,PORT]... - List of ports to listen on. (Default: 7)
  tcp_backlog=TCP_BACKLOG - Backlog value to use for listen(). (Default: 64)

event::libevent
  threads=THREADS - Number of event threads to use. (Default: 1)
  thread_stack_size=STACKSIZE - Size of the stack for threads. (Default: 0)
  
...

There is also some text in the help output that describes how module options can be specified. Note that help sections are only displayed for those modules that are told to load. If we also wanted to see echo::server::udp options, we would need to include that module name on the command line as well.