Feb 152017
 

Introduction

During C++ development for the Windows OS one often has to convert a stream from UTF-8 (e. g. an XML-File) to UTF-16 (standard encoding for almost all OS APIs).

Usually I do this by creating a std::wistream and imbue() it with a conversion facet like this:

Pretty standard, but what if you are just given an std::istream (e. g. as parameter of a function you can’t or don’t want to change to std::wistream)?

I found two solutions that are almost as straightforward to use as imbue(). The first one requires only standard C++11, the second one utilizes Boost.IoStreams.

Standard C++11 Solution

I just provide a code example with explanations in the comments. You may look up std::wbuffer_convert for further details.

Boost.IoStreams Solution

What is Boost.IoStreams? It is basically a library to easily create custom streams that are compatible with standard C++ streams.

Using Boost.IoStreams the code goes like this:

Some explanations:

  • Line 9 creates a “device” which is a concept that exists solely in the boost::iostreams library. This type is just used in the following line to create the actual standard-compatible stream type.
  • Line 18 creates an instance of that stream which just wraps the std::istream parameter. The boost::ref is required because Boost.IoStreams requires devices to be copyable (a standard stream is just a model of device in terms of Boost.IoStreams). Standard streams are by definition non-copyable, so we must work around that by using boost::ref which is a copyable reference-wrapper (see Design Rationale of Boost.IoStreams).

Conclusion

Both the standard C++11 and the Boost.IoStreams solution are pretty straightforward to use. There are basically just one or two additional code lines to wrap a wide stream around the narrow stream (not counting the includes and the using-statements)!

I prefer the standard C++11 solution because it has less dependencies. The Boost.IoStreams solution might be useful in a context where one already works with Boost.IoStreams though.