Last time we looked at how to write a server application in C++, and call it from Python. This time, we’ll do things the other way around: implementing a service in Python, and calling from C++.

Python server

Unlike our C++ server example, Thrift doesn’t help us out so much — we don’t get a helpfully pre-written skeleton, and we have to do all of the work ourselves. That said, it’s still pretty straightforward.

Essentially all we need to do is to create a class, derived from the service.Iface class that Thrift writes for us, to represent our service — defining methods as required by the service.

As a reminder here is our Thrift service definition file (LoggerService.thrift)

namespace py LoggerPy
namespace cpp LoggerCpp

exception LoggerException
{
    1: i32 error_code,
    2: string error_description
}

service Logger
{
    oneway void timestamp (1: string filename)
    string get_last_log_entry (1: string filename) throws (1: LoggerException error)
    void write_log (1: string filename, 2: string message) throws (1: LoggerException   error)
    i32 get_log_size (1: string filename) throws (1: LoggerException error)
}

So our class needs to contain definitions for the four methods — plus any initialization code we need.

class LoggerHandler (Logger.Iface):
    def __init__(self):
        # Initialization...
        pass

    def timestamp(self, filename):
        try:
            with open (filename, 'a') as f:
                print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
                 f.close()
             except IOError as e:
                 err = LoggerException()
                 err.error_code = 1
                 err.error_description = "Could not open file " & filename
                 raise err

    def get_last_log_entry(self, filename):
        try:
            with open (filename, 'r') as f:
                 last = None
                 for last in (line for line in f if line.rstrip('\n')):
                     pass
                 f.close()
             return last.rstrip('\n')

        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " & filename
            raise err

    def write_log(self, filename, message):
        try:
            with open (filename, 'a') as f:
            print(message, file=f)
            f.close()
        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " & filename
            raise err

    def get_log_size(self, filename):
        return os.path.getsize(filename)

Note that unlike our C++ example, the Python implementation of get_last_log_entry

uses a conventional return for the string; rather than the pass-by-reference technique we needed to use in C++.

Then all that we need to do, is to import the various files that Thrift has written for us, which define the underlaying functionality; and start the service running.

This next code snippet is more or less boilerplate code — though obviously you will need to change the references to the Service and Namespace names.

import sys
sys.path.append('gen-py')

from LoggerPy import Logger
from LoggerPy.ttypes import *
from LoggerPy.constants import *

from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer

class LoggerHandler (Logger.Iface):
    ...

    handler = LoggerHandler()
    processor = Logger.Processor(handler)
    transport = TSocket.TServerSocket(port=9090)
    tfactory = TTransport.TBufferedTransportFactory()
    pfactory = TBinaryProtocol.TBinaryProtocolFactory()
    
    server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
    
    server.serve()

So the final code looks like this:

from __future__ import print_function
import sys
import os
sys.path.append('gen-py')

from LoggerPy import Logger
from LoggerPy.ttypes import *
from LoggerPy.constants import *

from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer

import datetime

class LoggerHandler (Logger.Iface):
    def __init__(self):
        # Initialization...
        pass
    
    def timestamp(self, filename):
        try:
            with open (filename, 'a') as f:
                print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
                f.close()
        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " & filename
            raise err

    def get_last_log_entry(self, filename):
        try:
            with open (filename, 'r') as f:
                last = None
                for last in (line for line in f if line.rstrip('\n')):
                    pass

                f.close()
                return last.rstrip('\n')

        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " & filename
            raise err

    def write_log(self, filename, message):
        try:
            with open (filename, 'a') as f:
                print(message, file=f)
                f.close()
        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " & filename
            raise err

    def get_log_size(self, filename):
        return os.path.getsize(filename)

        handler = LoggerHandler()
        processor = Logger.Processor(handler)
        transport = TSocket.TServerSocket(port=9090)
        tfactory = TTransport.TBufferedTransportFactory()
        pfactory = TBinaryProtocol.TBinaryProtocolFactory()

        server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)

        server.serve()

C++ Client

Now for the C++ Client.

This is pretty much exactly a C++ conversion of the previous Python client application. The only significant difference is that we’ll use boost to create the socket / transport…

#include <iostream>

#include "Logger.h"

#include <thrift/transport/TSocket.h>
#include <thrift/transport/TBufferTransports.h>
#include <thrift/protocol/TBinaryProtocol.h>

using namespace apache::thrift;
using namespace apache::thrift::protocol;
using namespace apache::thrift::transport;

using namespace LoggerCpp;

int main(int argc, char **argv)
{
    char logfile[]="logfile.log";
    std::string line;

    boost::shared_ptr<TSocket> socket(new TSocket("localhost", 9090));
    boost::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
    boost::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));

    LoggerClient client(protocol);

    try
    {
        transport->open();

        client.timestamp(logfile);
        std::cout << "Logged timestamp to log file" << std::endl;

        client.write_log(logfile, "This is a message that I am writing to the log");
        client.timestamp(logfile);

        client.get_last_log_entry(line, logfile);
        std::cout << "Last line of the log file is: " << line << std::endl;
        std::cout << "Size of log file is: " << client.get_log_size(logfile) << " bytes" << std::endl;

        transport->close();
    }

    catch (TTransportException e)
    {
        std::cout << "Error starting client" << std::endl;
    }

    catch (LoggerException e)
    {
        std::cout << e.error_description << std::endl;
    }

    return 0;
}

Note once again, that we’re back with having to use the pass-by-reference for the string that’s returned by get_last_log_entry.

To build the C++ client, we need (essentially) the same build process as we used for the server. Since we already have a CMake file to hand, let’s use that to generate the build-script for the client.

cmake_minimum_required(VERSION 3.5)

project(LoggerService)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H")

set(THRIFT_DIR "/usr/local/include/thrift")
set(BOOST_DIR "/usr/local/Cellar/boost/1.60.0_2/include/")

include_directories(${THRIFT_DIR} ${BOOST_DIR} ${CMAKE_SOURCE_DIR})
link_directories(/usr/local/lib)

set(BASE_SOURCE_FILES Logger.cpp Logger_Service_types.cpp Logger_Service_constants.cpp)
set(SERVER_FILES LoggerServer.cpp)
set(CLIENT_FILES LoggerClient.cpp)

add_executable(LoggerServer ${SERVER_FILES} ${BASE_SOURCE_FILES})
add_executable(LoggerClient ${CLIENT_FILES} ${BASE_SOURCE_FILES})

target_link_libraries(LoggerServer thrift)
target_link_libraries(LoggerClient thrift)

In comparison to the previous file, all that we need to do is add the CLIENT_FILES definition, and add (and link) the LoggerClient executable. If you’re only building the server, then of course you don’t need the LoggerServer lines…

ThriftPy

There is one other approach that you can take when using Thrift with Python, and that is to use the ThriftPy library.

ThriftPy aims to take a more pythonic approach to Thrift. Unlike the conventional approach — when using ThriftPy you don’t need to run thrift —gen py my file.thrift to statically generate the Python code: rather you can load the .thrift file into your Python program, and dynamically create the support functions. It also removes the requirement for the (binary) Thrift libraries to be installed on the machine running the ThriftPy code. The only installation step required is pip install thriftpy.

For example, to use ThriftPy with our LoggerService.thrift file we use:

import thriftpy

logger_service = thriftpy.load("LoggerService.thrift", "loggerservice_thrift")

from thriftpy.rpc import make_client

client = make_client(logger_service.Logger, 'localhost', 9090)

...

From here — we simply implement the client application in exactly the same way as we did previously in Python.

The only change is if we’re using exception handling — where instead of catching our LoggerExceptions as Thrift.TException — we need to use thriftpy.thrift.TException.

So the full code for the ThriftPy client becomes:

import thriftpy

logger_service = thriftpy.load("../LoggerService.thrift", "loggerservice_thrift")

from thriftpy.rpc import make_client

try:
    client = make_client(logger_service.Logger, 'localhost', 9090)

    logfile = "logofile.log"

    client.timestamp(logfile)
    print ("Logged timestamp to log file")

    client.write_log(logfile, "This is a message that I am writing to the log")
    client.timestamp(logfile)

    print ("Last line of log file is: %s" % (client.get_last_log_entry(logfile)))
    print ("Size of log file is: %d bytes" % client.get_log_size(logfile))

except thriftpy.transport.TTransportException, e:
    print ("Error starting client")
except thriftpy.Thrift.TApplicationException, e:
    print ("%s" % (e))
except thriftpy.thrift.TException, e:
    print ("Error: %d %s" % (e.error_code, e.error_description))

Implementing the server can be don win a similar manner.

Once again we load our .thrift file at runtime, create a server object, and start it running.

The slight difference when using ThriftPy to using the more Thrift generated Python code is that instead of implementing our server methods in a class that inherits fromLogger.Iface — we put them in a Dispatcher class which inherits from the Python base-object object. This class is then passed as a parameter to make_server.

For example:

import thriftpy
logger_service = thriftpy.load("../LoggerService.thrift", module_name="LoggerService_thrift")

from thriftpy.rpc import make_server

class Dispatcher(object):
    ...

    server = make_server(logger_service.Logger, Dispatcher(), 'localhost', 9090)
    server.serve()

Note that there’s no requirement for the Dispatcher to be called Dispatcher() — you can name it anything you like.

The implementation of the methods within Dispatcher() are identical to the previous method.

So the full version of the ThriftPy server would look like this:

from __future__ import print_function
import os
import datetime

import thriftpy
logger_service = thriftpy.load("../LoggerService.thrift", module_name="LoggerService_thrift")

from thriftpy.rpc import make_server

class Dispatcher(object):
    def timestamp(self, filename):
        try:
            with open (filename, 'a') as f:
                print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
                f.close()
        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " + filename
            raise err

    def get_last_log_entry(self, filename):
        try:
            with open (filename, 'r') as f:
                last = None
                for last in (line for line in f if line.rstrip('\n')):
                    pass
             f.close()
             return last.rstrip('\n')
        except IOError as e:
            err.error_code = 1
            err.error_description = "Could not open file " + filename
            raise err

    def write_log(self, filename, message):
        try:
            with open (filename, 'a') as f:
                print(message, file=f)
                f.close()
        except IOError as e:
            err = LoggerException()
            err.error_code = 1
            err.error_description = "Could not open file " + filename
            raise err

    def get_log_size(self, filename):
        return os.path.getsize(filename)

server = make_server(logger_service.Logger, Dispatcher(), 'localhost', 9090)
server.serve()

Whether or not you think using ThriftPy is easier or not, is up to you — but if you want your code to be implemented in pure Python, or prefer the more pythonic approach to doing things, then it’s certainly worth a look.

I’ve tested my simple LoggerService example with all combinations of clients and servers implemented using all three of the methods described; and regardless of the implementation, all three approaches are fully interoperable with each other: so, as the saying goes, you pays your money and you takes your choice…

I hope that’s been interesting, and/or useful. If you have any questions of comments then please let me know.

If you want to try this for yourself, then the complete source code for both client & server, in both Python, ThriftPy, and C++ can be found at: https://github.com/Auctoris/ThriftDemo

---

Filed under: C++,Computing,Python,Uncategorised - @ August 19, 2016 12:53