Last time we looked at how to write a server application in C++, and call it from Python. This time, we’ll do things the other way around: implementing a service in Python, and calling from C++.
Python server
Unlike our C++ server example, Thrift doesn’t help us out so much — we don’t get a helpfully pre-written skeleton, and we have to do all of the work ourselves. That said, it’s still pretty straightforward.
Essentially all we need to do is to create a class, derived from the service.Iface class that Thrift writes for us, to represent our service – defining methods as required by the service.
As a reminder here is our Thrift service definition file (LoggerService.thrift)
1namespace py LoggerPy
2namespace cpp LoggerCpp
3
4exception LoggerException
5{
6 1: i32 error_code,
7 2: string error_description
8}
9
10service Logger
11{
12 oneway void timestamp (1: string filename)
13 string get_last_log_entry (1: string filename) throws (1: LoggerException error)
14 void write_log (1: string filename, 2: string message) throws (1: LoggerException error)
15 i32 get_log_size (1: string filename) throws (1: LoggerException error)
16}
So our class needs to contain definitions for the four methods — plus any initialization code we need.
1class LoggerHandler (Logger.Iface):
2 def __init__(self):
3 # Initialization...
4 pass
5
6 def timestamp(self, filename):
7 try:
8 with open (filename, 'a') as f:
9 print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
10 f.close()
11 except IOError as e:
12 err = LoggerException()
13 err.error_code = 1
14 err.error_description = "Could not open file " & filename
15 raise err
16
17 def get_last_log_entry(self, filename):
18 try:
19 with open (filename, 'r') as f:
20 last = None
21 for last in (line for line in f if line.rstrip('\n')):
22 pass
23 f.close()
24 return last.rstrip('\n')
25
26 except IOError as e:
27 err = LoggerException()
28 err.error_code = 1
29 err.error_description = "Could not open file " & filename
30 raise err
31
32 def write_log(self, filename, message):
33 try:
34 with open (filename, 'a') as f:
35 print(message, file=f)
36 f.close()
37 except IOError as e:
38 err = LoggerException()
39 err.error_code = 1
40 err.error_description = "Could not open file " & filename
41 raise err
42
43 def get_log_size(self, filename):
44 return os.path.getsize(filename)
Note that unlike our C++ example, the Python implementation of get_last_log_entry uses a conventional return for the string; rather than the pass-by-reference technique we needed to use in C++.
Then all that we need to do, is to import the various files that Thrift has written for us, which define the underlaying functionality; and start the service running.
This next code snippet is more or less boilerplate code — though obviously you will need to change the references to the Service and Namespace names.
1import sys
2sys.path.append('gen-py')
3
4from LoggerPy import Logger
5from LoggerPy.ttypes import *
6from LoggerPy.constants import *
7
8from thrift.transport import TSocket
9from thrift.transport import TTransport
10from thrift.protocol import TBinaryProtocol
11from thrift.server import TServer
12
13class LoggerHandler (Logger.Iface):
14 ...
15
16 handler = LoggerHandler()
17 processor = Logger.Processor(handler)
18 transport = TSocket.TServerSocket(port=9090)
19 tfactory = TTransport.TBufferedTransportFactory()
20 pfactory = TBinaryProtocol.TBinaryProtocolFactory()
21
22 server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
23
24 server.serve()
So the final code looks like this:
1from __future__ import print_function
2import sys
3import os
4sys.path.append('gen-py')
5
6from LoggerPy import Logger
7from LoggerPy.ttypes import *
8from LoggerPy.constants import *
9
10from thrift.transport import TSocket
11from thrift.transport import TTransport
12from thrift.protocol import TBinaryProtocol
13from thrift.server import TServer
14
15import datetime
16
17class LoggerHandler (Logger.Iface):
18 def __init__(self):
19 # Initialization...
20 pass
21
22 def timestamp(self, filename):
23 try:
24 with open (filename, 'a') as f:
25 print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
26 f.close()
27 except IOError as e:
28 err = LoggerException()
29 err.error_code = 1
30 err.error_description = "Could not open file " & filename
31 raise err
32
33 def get_last_log_entry(self, filename):
34 try:
35 with open (filename, 'r') as f:
36 last = None
37 for last in (line for line in f if line.rstrip('\n')):
38 pass
39
40 f.close()
41 return last.rstrip('\n')
42
43 except IOError as e:
44 err = LoggerException()
45 err.error_code = 1
46 err.error_description = "Could not open file " & filename
47 raise err
48
49 def write_log(self, filename, message):
50 try:
51 with open (filename, 'a') as f:
52 print(message, file=f)
53 f.close()
54 except IOError as e:
55 err = LoggerException()
56 err.error_code = 1
57 err.error_description = "Could not open file " & filename
58 raise err
59
60 def get_log_size(self, filename):
61 return os.path.getsize(filename)
62
63 handler = LoggerHandler()
64 processor = Logger.Processor(handler)
65 transport = TSocket.TServerSocket(port=9090)
66 tfactory = TTransport.TBufferedTransportFactory()
67 pfactory = TBinaryProtocol.TBinaryProtocolFactory()
68
69 server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)
70
71 server.serve()
C++ Client
Now for the C++ Client.
This is pretty much exactly a C++ conversion of the previous Python client application. The only significant difference is that we’ll use boost to create the socket / transport…
1#include <iostream>
2
3#include "Logger.h"
4
5#include <thrift/transport/TSocket.h>
6#include <thrift/transport/TBufferTransports.h>
7#include <thrift/protocol/TBinaryProtocol.h>
8
9using namespace apache::thrift;
10using namespace apache::thrift::protocol;
11using namespace apache::thrift::transport;
12
13using namespace LoggerCpp;
14
15int main(int argc, char **argv)
16{
17 char logfile[]="logfile.log";
18 std::string line;
19
20 boost::shared_ptr<TSocket> socket(new TSocket("localhost", 9090));
21 boost::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
22 boost::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));
23
24 LoggerClient client(protocol);
25
26 try
27 {
28 transport->open();
29
30 client.timestamp(logfile);
31 std::cout << "Logged timestamp to log file" << std::endl;
32
33 client.write_log(logfile, "This is a message that I am writing to the log");
34 client.timestamp(logfile);
35
36 client.get_last_log_entry(line, logfile);
37 std::cout << "Last line of the log file is: " << line << std::endl;
38 std::cout << "Size of log file is: " << client.get_log_size(logfile) << " bytes" << std::endl;
39
40 transport->close();
41 }
42
43 catch (TTransportException e)
44 {
45 std::cout << "Error starting client" << std::endl;
46 }
47
48 catch (LoggerException e)
49 {
50 std::cout << e.error_description << std::endl;
51 }
52
53 return 0;
54}
Note once again, that we’re back with having to use the pass-by-reference for the string that’s returned by get_last_log_entry.
To build the C++ client, we need (essentially) the same build process as we used for the server. Since we already have a CMake file to hand, let’s use that to generate the build-script for the client.
1cmake_minimum_required(VERSION 3.5)
2
3project(LoggerService)
4set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H")
5
6set(THRIFT_DIR "/usr/local/include/thrift")
7set(BOOST_DIR "/usr/local/Cellar/boost/1.60.0_2/include/")
8
9include_directories(${THRIFT_DIR} ${BOOST_DIR} ${CMAKE_SOURCE_DIR})
10link_directories(/usr/local/lib)
11
12set(BASE_SOURCE_FILES Logger.cpp Logger_Service_types.cpp Logger_Service_constants.cpp)
13set(SERVER_FILES LoggerServer.cpp)
14set(CLIENT_FILES LoggerClient.cpp)
15
16add_executable(LoggerServer ${SERVER_FILES} ${BASE_SOURCE_FILES})
17add_executable(LoggerClient ${CLIENT_FILES} ${BASE_SOURCE_FILES})
18
19target_link_libraries(LoggerServer thrift)
20target_link_libraries(LoggerClient thrift)
In comparison to the previous file, all that we need to do is add the CLIENT_FILES definition, and add (and link) the LoggerClient executable. If you’re only building the server, then of course you don’t need the LoggerServer lines…
ThriftPy
There is one other approach that you can take when using Thrift with Python, and that is to use the ThriftPy library.
ThriftPy aims to take a more pythonic approach to Thrift. Unlike the conventional approach — when using ThriftPy you don’t need to run thrift —gen py my file.thrift to statically generate the Python code: rather you can load the .thrift file into your Python program, and dynamically create the support functions. It also removes the requirement for the (binary) Thrift libraries to be installed on the machine running the ThriftPy code. The only installation step required is pip install thriftpy.
For example, to use ThriftPy with our LoggerService.thrift file we use:
1import thriftpy
2
3logger_service = thriftpy.load("LoggerService.thrift", "loggerservice_thrift")
4
5from thriftpy.rpc import make_client
6
7client = make_client(logger_service.Logger, 'localhost', 9090)
8
9...
From here — we simply implement the client application in exactly the same way as we did previously in Python.
The only change is if we’re using exception handling — where instead of catching our LoggerExceptions as Thrift.TException – we need to use thriftpy.thrift.TException.
So the full code for the ThriftPy client becomes:
1import thriftpy
2
3logger_service = thriftpy.load("../LoggerService.thrift", "loggerservice_thrift")
4
5from thriftpy.rpc import make_client
6
7try:
8 client = make_client(logger_service.Logger, 'localhost', 9090)
9
10 logfile = "logofile.log"
11
12 client.timestamp(logfile)
13 print ("Logged timestamp to log file")
14
15 client.write_log(logfile, "This is a message that I am writing to the log")
16 client.timestamp(logfile)
17
18 print ("Last line of log file is: %s" % (client.get_last_log_entry(logfile)))
19 print ("Size of log file is: %d bytes" % client.get_log_size(logfile))
20
21except thriftpy.transport.TTransportException, e:
22 print ("Error starting client")
23except thriftpy.Thrift.TApplicationException, e:
24 print ("%s" % (e))
25except thriftpy.thrift.TException, e:
26 print ("Error: %d %s" % (e.error_code, e.error_description))
Implementing the server can be don win a similar manner.
Once again we load our .thrift file at runtime, create a server object, and start it running.
The slight difference when using ThriftPy to using the more Thrift generated Python code is that instead of implementing our server methods in a class that inherits fromLogger.Iface – we put them in a Dispatcher class which inherits from the Python base-object object. This class is then passed as a parameter to make_server.
For example:
1import thriftpy
2logger_service = thriftpy.load("../LoggerService.thrift", module_name="LoggerService_thrift")
3
4from thriftpy.rpc import make_server
5
6class Dispatcher(object):
7 ...
8
9 server = make_server(logger_service.Logger, Dispatcher(), 'localhost', 9090)
10 server.serve()
Note that there’s no requirement for the Dispatcher to be called Dispatcher() – you can name it anything you like.
The implementation of the methods within Dispatcher() are identical to the previous method.
So the full version of the ThriftPy server would look like this:
1from __future__ import print_function
2import os
3import datetime
4
5import thriftpy
6logger_service = thriftpy.load("../LoggerService.thrift", module_name="LoggerService_thrift")
7
8from thriftpy.rpc import make_server
9
10class Dispatcher(object):
11 def timestamp(self, filename):
12 try:
13 with open (filename, 'a') as f:
14 print(datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y"), file=f)
15 f.close()
16 except IOError as e:
17 err = LoggerException()
18 err.error_code = 1
19 err.error_description = "Could not open file " + filename
20 raise err
21
22 def get_last_log_entry(self, filename):
23 try:
24 with open (filename, 'r') as f:
25 last = None
26 for last in (line for line in f if line.rstrip('\n')):
27 pass
28 f.close()
29 return last.rstrip('\n')
30 except IOError as e:
31 err.error_code = 1
32 err.error_description = "Could not open file " + filename
33 raise err
34
35 def write_log(self, filename, message):
36 try:
37 with open (filename, 'a') as f:
38 print(message, file=f)
39 f.close()
40 except IOError as e:
41 err = LoggerException()
42 err.error_code = 1
43 err.error_description = "Could not open file " + filename
44 raise err
45
46 def get_log_size(self, filename):
47 return os.path.getsize(filename)
48
49server = make_server(logger_service.Logger, Dispatcher(), 'localhost', 9090)
50server.serve()
Whether or not you think using ThriftPy is easier or not, is up to you – but if you want your code to be implemented in pure Python, or prefer the more pythonic approach to doing things, then it’s certainly worth a look.
I’ve tested my simple LoggerService example with all combinations of clients and servers implemented using all three of the methods described; and regardless of the implementation, all three approaches are fully interoperable with each other: so, as the saying goes, you pays your money and you takes your choice…
I hope that’s been interesting, and/or useful. If you have any questions of comments then please let me know.
If you want to try this for yourself, then the complete source code for both client & server, in both Python, ThriftPy, and C++ can be found at: https://github.com/Auctoris/ThriftDemo