RDKit: boost_adaptbx::python::streambuf Class Reference

| RDKit Open-source cheminformatics and machine learning.

Loading...
Searching...
No Matches
Classes | Public Types | Public Member Functions | Static Public Member Functions | Static Public Attributes | List of all members
boost_adaptbx::python::streambuf Class Reference

A stream buffer getting data from and putting data into a Python file object. More...

`#include <python_streambuf.h>`

Inheritance diagram for boost_adaptbx::python::streambuf:

## Classes
class | istream
class | ostream

## Public Types
typedef base_t::char_type | char_type
typedef base_t::int_type | int_type
typedef base_t::pos_type | pos_type
typedef base_t::off_type | off_type
typedef base_t::traits_type | traits_type

## Public Member Functions
| streambuf (bp::object &python_file_obj, std::size_t buffer_size_=0)
| Construct from a Python file object.
| streambuf (bp::object &python_file_obj, char mode, std::size_t buffer_size_=0)
| constructor to enforce a mode (binary or text)
| ~streambuf () override
| Mundane destructor freeing the allocated resources.
std::streamsize | showmanyc () override
| C.f. C++ standard section 27.5.2.4.3.
int_type | underflow () override
| C.f. C++ standard section 27.5.2.4.3.
int_type | overflow (int_type c=traits_type_eof()) override
| C.f. C++ standard section 27.5.2.4.5.
int | sync () override
| Update the python file to reflect the state of this stream buffer.
pos_type | seekoff (off_type off, std::ios_base::seekdir way, std::ios_base::openmode which=std::ios_base::in|std::ios_base::out) override
| C.f. C++ standard section 27.5.2.4.2.
pos_type | seekpos (pos_type sp, std::ios_base::openmode which=std::ios_base::in|std::ios_base::out) override
| C.f. C++ standard section 27.5.2.4.2.

## Static Public Member Functions
static int | traits_type_eof ()

## Static Public Attributes
static const std::size_t | default_buffer_size = 1024
| The default size of the read and write buffer.

## Detailed Description

A stream buffer getting data from and putting data into a Python file object.

The aims are as follow:

- Given a C++ function acting on a standard stream, e.g.

  void read_inputs(std::istream& input) {
  ...
  input >> something >> something_else;
  }

  and given a piece of Python code which creates a file-like object, to be able to pass this file object to that C++ function, e.g.

  import gzip
  gzip_file_obj = gzip.GzipFile(...)
  read_inputs(gzip_file_obj)

  and have the standard stream pull data from and put data into the Python file object.

- When Python `read_inputs()` returns, the Python object is able to continue reading or writing where the C++ code left off.
- Operations in C++ on mere files should be competitively fast compared to the direct use of `std::fstream`.

Motivation

- the standard Python library offer of file-like objects (files, compressed files and archives, network, ...) is far superior to the offer of streams in the C++ standard library and Boost C++ libraries.
- i/o code involves a fair amount of text processing which is more efficiently prototyped in Python but then one may need to rewrite a time-critical part in C++, in as seamless a manner as possible.

Usage

This is 2-step:

- a trivial wrapper function

  using boost_adaptbx::python::streambuf;
  void read_inputs_wrapper(streambuf& input)
  {
  streambuf::istream is(input);
  read_inputs(is);
  }
  def("read_inputs", read_inputs_wrapper);
  boost_adaptbx::python::streambuf::istream
  Definition python_streambuf.h:509
  boost_adaptbx::python::streambuf
  A stream buffer getting data from and putting data into a Python file object.
  Definition python_streambuf.h:110
  boost_adaptbx::python::streambuf::streambuf
  streambuf(bp::object &python_file_obj, std::size_t buffer_size_=0)
  Construct from a Python file object.
  Definition python_streambuf.h:137

  which has to be written every time one wants a Python binding for such a C++ function.

- the Python side

  from boost.python import streambuf
  read_inputs(streambuf(python_file_obj=obj, buffer_size=1024))
  boost
  Definition RDLog.h:24

  `buffer_size` is optional. See also: `default_buffer_size`

Note: references are to the C++ standard (the numbers between parentheses at the end of references are margin markers).

Definition at line 110 of file python_streambuf.h.

## Member Typedef Documentation

## ◆ char_type

typedef base_t::char_type boost_adaptbx::python::streambuf::char_type

Definition at line 119 of file python_streambuf.h.

## ◆ int_type

typedef base_t::int_type boost_adaptbx::python::streambuf::int_type

Definition at line 120 of file python_streambuf.h.

## ◆ off_type

typedef base_t::off_type boost_adaptbx::python::streambuf::off_type

Definition at line 122 of file python_streambuf.h.

## ◆ pos_type

typedef base_t::pos_type boost_adaptbx::python::streambuf::pos_type

Definition at line 121 of file python_streambuf.h.

## ◆ traits_type

typedef base_t::traits_type boost_adaptbx::python::streambuf::traits_type

Definition at line 123 of file python_streambuf.h.

## Constructor & Destructor Documentation

## ◆ streambuf() [1/2]

boost_adaptbx::python::streambuf::streambuf | ( | bp::object & | python_file_obj,
|  | std::size_t | buffer_size_ = 0 )
| inline

Construct from a Python file object.

if buffer_size is 0 the current default_buffer_size is used.

Definition at line 137 of file python_streambuf.h.

References default_buffer_size, and TEST_ASSERT.

Referenced by boost_adaptbx::python::streambuf::istream::istream(), boost_adaptbx::python::streambuf::ostream::ostream(), boost_adaptbx::python::streambuf::ostream::ostream(), and streambuf().

## ◆ streambuf() [2/2]

boost_adaptbx::python::streambuf::streambuf | ( | bp::object & | python_file_obj,
|  | char | mode,
|  | std::size_t | buffer_size_ = 0 )
| inline

constructor to enforce a mode (binary or text)

yeah, is redundant, but it is somehow natural to do "s"

Definition at line 191 of file python_streambuf.h.

References CHECK_INVARIANT, and streambuf().

## ◆ ~streambuf()

boost_adaptbx::python::streambuf::~streambuf | ( |  | ) |
| inlineoverride

Mundane destructor freeing the allocated resources.

Definition at line 235 of file python_streambuf.h.

## Member Function Documentation

## ◆ overflow()

int_type boost_adaptbx::python::streambuf::overflow | ( | int_type | c = traits_type_eof() | ) |
| inlineoverride

C.f. C++ standard section 27.5.2.4.5.

Definition at line 282 of file python_streambuf.h.

Referenced by seekoff(), and sync().

## ◆ seekoff()

pos_type boost_adaptbx::python::streambuf::seekoff | ( | off_type | off,
|  | std::ios_base::seekdir | way,
|  | std::ios_base::openmode | which = std::ios_base::in | std::ios_base::out )
| inlineoverride

C.f. C++ standard section 27.5.2.4.2.

This implementation is optimised to look whether the position is within the buffers, so as to avoid calling Python seek or tell. It is important for many applications that the overhead of calling into Python is avoided as much as possible (e.g. parsers which may do a lot of backtracking)

Definition at line 365 of file python_streambuf.h.

References overflow(), and underflow().

Referenced by seekpos().

## ◆ seekpos()

pos_type boost_adaptbx::python::streambuf::seekpos | ( | pos_type | sp,
|  | std::ios_base::openmode | which = std::ios_base::in | std::ios_base::out )
| inlineoverride

C.f. C++ standard section 27.5.2.4.2.

Definition at line 428 of file python_streambuf.h.

References seekoff().

## ◆ showmanyc()

std::streamsize boost_adaptbx::python::streambuf::showmanyc | ( |  | ) |
| inlineoverride

C.f. C++ standard section 27.5.2.4.3.

It is essential to override this virtual function for the stream member function readsome to work correctly (c.f. 27.6.1.3, alinea 30)

Definition at line 245 of file python_streambuf.h.

References underflow().

## ◆ sync()

int boost_adaptbx::python::streambuf::sync | ( |  | ) |
| inlineoverride

Update the python file to reflect the state of this stream buffer.

Empty the write buffer into the Python file object and set the seek position of the latter accordingly (C++ standard section 27.5.2.4.2). If there is no write buffer or it is empty, but there is a non-empty read buffer, set the Python file object seek position to the seek position in that read buffer.

Definition at line 338 of file python_streambuf.h.

References overflow().

## ◆ traits_type_eof()

static int boost_adaptbx::python::streambuf::traits_type_eof | ( |  | ) |
| inlinestatic

Definition at line 126 of file python_streambuf.h.

## ◆ underflow()

int_type boost_adaptbx::python::streambuf::underflow | ( |  | ) |
| inlineoverride

C.f. C++ standard section 27.5.2.4.3.

Definition at line 255 of file python_streambuf.h.

Referenced by seekoff(), and showmanyc().

## Member Data Documentation

## ◆ default_buffer_size

const std::size_t boost_adaptbx::python::streambuf::default_buffer_size = 1024
| static

The default size of the read and write buffer.

They are respectively used to buffer data read from and data written to the Python file object. It can be modified from Python.

Definition at line 132 of file python_streambuf.h.

Referenced by streambuf().

The documentation for this class was generated from the following file:

- python_streambuf.h