TextEngine brainstorm - Cinder Forum ARCHIVE

Loading...

Cinder Forum ARCHIVE

Recent Topics
All Forums
- Search All
- Tags
- Author
- Advanced Search
Search
New Topic

Developing Cinder

TextEngine brainstorm

in Developing Cinder • 8 years ago

This is a continuation of recent discussion, wherein Andrew mentioned his design ideas for a TextEngine. The gyst of it to me is that could use a more extendable glyph manager, which is quite apparent by looking at the number of #defines in the TextureFont implementation. The problem is that every platform handles glyphs differently, and none of it is pretty. I'll repeat his description before going on:

Currently my thought is that we should design a TextEngine class which implements acquiring glyphs, placing glyphs, measuring strings, instantiating Fonts, etc. So we'd have a TextEngineCoreText, TextEngineFreeType, TextEngineDirectWrite, etc. This means that Fonts are a "child" of a particular TextEngine as well. For convenience I'm inclined toward creating a singleton that is the default TextEngine which you get when you don't specify one. So you can easily enable say, FreeType globally from your app's setup and then not worry about it anymore. But if you had some particular need to do so, you could provide an instance of TextEngine to the Font constructor or something along those lines.

The important design features I get out of this are:

The implementation can possibly be defined at runtime
Font's will be managed by the TextEngine, not the app.
There will be one global TextEngine, while more can be created for special circumstances

I've been looking at the TextureFont implementation for Cocoa quite a bit over the last couple weeks, so I figure I can plunge in and prototype this in a CinderBlock for OS X. But first, some questions:

Is there only one Font per name/size pair?
Will the TextEngine manage Textures?

i.e. a Texture Atlas, such as the one in safetydank's fork

Does this affect TextureFont's forward facing API?
I don't think it is too early to start considering techniques like SDF or the programmable pipeline in general. The current version of TextureFont is limited here, but maybe it would be appropriate for the proposed TextEngine to also generate vertex information in a generic way.

Well, this is an initial list of questions before writing any code. :)

1

Replies(35)

Re: TextEngine brainstorm

8 years ago

Hey - I think your summary is a good one. Here's my take on some of the questions you've raised:

• I think there is one Font per name/size pair, in part because this maps to how most the platform APIs themselves work.

• I think the TextEngine is definitely independent of Textures. To my mind a class like gl::TextureFont utilizing the TextEngine in order to create its Textures is its own "business" - the TextEngine is agnostic about that sort of thing.

• I agree on the SDF sentiment. I would imagine this would be something like gl::SdfTextureFont or similar. It's conceivable we'd create a base class common to TextureFont as well, or possibly derive SdfTextureFont from TextureFont.

A very rough sketch of the API might go like this:

class TextEngine {

public:

FontRef createFont( string name, float size );

Surface renderGlyph( FontRef font, uint16_t glyph );

Surface renderString( FontRef font, string utf8, Rectf bounds );

Vec2f measureString( FontRef font, string utf8 );

vector<uint16_t,Vec2f> measureGlyphs( FontRef font, string utf8, Rectf bounds );

};

class TextEngineCoreText : public TextEngine {

}; // etc

The idea would be to put enough functionality into TextEngine that all platform-specific code in TextBox and gl::TextureFont goes away.

Also, if we can solve this design, I would imagine this would be part of the Cinder core rather than a CinderBlock.

-Andrew

Re: TextEngine brainstorm

8 years ago

I started some work on this today, mainly because I wanted to get FreeType up and running on Windows asap. I'm doing the work in the textengine branch here:

https://github.com/safetydank/Cinder/tree/textengine

It's pretty rough right now but I hope to have Windows and Android rendering working in the next couple of days.

paul.houx

Reageren: TextEngine brainstorm

8 years ago

Hi,

What I think is missing from the list are classes to tokenize std::strings and std::wstrings, to easily determine where line breaks could go. Another thing that's missing is some kind of markup support, so it becomes easy to specify and mix different font types, colors, etc. Perhaps using rich-text-format.

I've succeeded in tokenizing strings for basic word wrapping, but it's not compatible with Asian or Arabic languages (or any language other than those using the western alphabet).

When I have finished working on the SDF "text engine", I will release the code on Github. It doesn't use any text functionality currently in Cinder though, but it works for me and might give you guys some ideas.

-Paul

Re: TextEngine brainstorm

8 years ago

@safetydank - very cool; looks like you've already made some great progress. I've only skimmed the implementation, but it looks really promising. One thing I did notice is that I think I steered us wrong with suggesting a pimpl on the Font. If we move to a FontRef (which I think we should, and you've already started implementation), a pimpl doesn't buy us anything relative to just implementing an abstract interface. So,

class Font {

};

class FontCoreText : public Font {

};

Besides a slightly simpler implementation this has the advantage of letting you use RTTI for an advanced use case where you need say, some Core Text-specific function or something.

So I have actually done a fair amount of work on the word wrapping problem, and you can find it in my svg branch. It's demoed in test/linebreakTest, although it's a bit hard to follow for the time being. Suffice it to say I'm pretty confident we can properly handle the larger problem of proper line breaks, though at the moment I think right-to-left languages are out of our depth. The relevant function is lineBreakUtf8().

I was considering trying to do TextureFont wrapped string rendering on Windows but I think I'll put this off for 0.8.5, particularly if we're thinking we want to attempt this TextEngine overhaul.

And Paul is right, an attributed string representation is probably a necessary ingredient to a more full featured text implementation.

-Andrew

Re: TextEngine brainstorm

8 years ago

I've switched the Font implementation to a virtual interface now.

The pimpl version had a couple of things going for it - it hid the implementation details from the header file (mostly platform specific stuff). And it enabled the implementation definitions to be written inline. You could also use RTTI with pimpl and abstract interfaces to get at the platform specific stuff, I wrote a templated Font::getImpl class for that purpose.

But in the end it's a tradeoff and I didn't feel too strongly either way. The virtual interface removes a level of indirection which is a good thing and is probably clearer to follow.

I have been using a simple line break algorithm for Freetype, adapted from Skia's TextBox - linebreak(). I know they're tricky to get right, it will be nice to have a unified solution in place.

I have FreeType and GDI+ TextureFont's barely working:

GDI+

FreeType:

Re: TextEngine brainstorm

8 years ago

FreeType rendering is slow at the moment. It's expensive to lookup glyph bounding box information, and FreeType doesn't do any internal caching. I'd like to cache the bounding box on every call to getGlyphBoundingBox, but it's declared as a const function and so I can't save it inside a member map.

Easiest solution for me would be to change the function signature to be non-const. But is there a better way around this?

Re: TextEngine brainstorm

8 years ago

What about something like a mutable isCached variable? I use a similar technique on the svg::Node::getBoundingBox() method.

-Andrew

paul.houx

Reageren: Re: TextEngine brainstorm

8 years ago

Thanks, now I finally understand the meaning of 'mutable'!

Re: TextEngine brainstorm

8 years ago

Hey, just looked at @safetydank's branch, looks like you're taking charge! Also looks like a great start. Some initial questions / comments:

Why is there a FontManager and TextEngine? I thought TextEngine's manage fonts
similar q; is there a need for a FontManager singleton?
There are currently TextureFont's in TextEngine - is this going away? I'm really curious what's going to happen with the TextureFont::Atlas in this new design.
About the pimpl and inline methods, I think this will make the files huge and less maintainable. It's nice having separate implementation files for separate implementations. :)
remark about TextureFont: it's so much clearer this way.

I'd like to get this working on Mac, although I've never used FreeType before. It looks like there are some headers missing, once I get some time to fix that maybe I can help on the mac side.

Cheers,

Rich

Re: TextEngine brainstorm

8 years ago

One other thing, it looks like the textengine branch is on top of safetydank/android, not cinder/master, which'll make it difficult if not impossible to make a pull request (unless the idea is to merge the entire android branch).

Re: TextEngine brainstorm

8 years ago

@andrewfb - using mutable worked nicely, cheers

@reakinator

FontManager is untouched from master at the moment, where it is also defined as a singleton (wraps a GDI context). Yes, its methods should probably be merged into TextEngine. But it also implements some platform specific methods that will have to be exposed via TextEngine too, probably by converting it from a pimpl to a virtual interface.
re:TextureFont - ideally we could write a common cacheGlyphs implementation that uses TextEngine methods only. At the moment each version of cacheGlyphs uses a lot of platform api calls not covered by TextEngine. So it's an ongoing problem to refactor this, I'm open to ideas.
Wait, whose idea was it to use a pimpl? :) Like many C++ style debates it comes down to personal preferences. I appreciate a minimal header and inline declarations, YMMV. When we switch to a virtual interface I'll split out the definitions.
FreeType is pretty easy to build, there's a Unix makefile for Mac OS X.
I've worked off the android branch because it already had the FreeType libraries set up (via setup-android script) and is suitable for ES2 testing. When it's ready for a pull request I'll merge the textengine code into a master branch, clean up the code/formatting and submit.

Re: TextEngine brainstorm

8 years ago

Updates

Merged FontManager into TextEngine
Made TextEngine use a virtual interface implementation

Looking at cacheGlyphs side-by-side, I think I can see how a common implementation would look. Still figuring out how to integrate that with the texture atlas.

TextEngineCoreText is in a broken state now. Hopefuly I'll be able to look at that over the weekend, unless someone gets there first :)

Re: TextEngine brainstorm

8 years ago

I'm looking at OS X now and have a question:

How does Xcode compile Font.cpp as a C++ source file when it uses some Objective-C messaging?

I thought you could only mix the languages in an Objective-C++ file but Font.cpp is labelled C++ in Xcode and seems to do some messaging to NSFontManager.

Is Font.cpp being compiled as Objective-C++ despite the Xcode labelling?

Re: TextEngine brainstorm

8 years ago

Ah, I found the source property pane in Xcode has Font.cpp set to Objective-C++. Confusing!

Re: TextEngine brainstorm

8 years ago

Updated, need to fix the origin offset calculation but OS X is almost working. I'm using texture font atlases on all three engines now.

Re: TextEngine brainstorm

8 years ago

Exxxcccellent. I've got a copy of your branch running on my mac. You're definitely moving alot faster than I can at the moment, can't believe you got to three platforms so quickly.

About the pimpl, the original idea was to make TextEngine one (not Font, since TextEngine manages fonts, so these can be handled well enough as shared_ptrs), which appears to be what you've done, although you have a static sInstance and not ivar mImpl. :). Are you planning to work in multiple instances? I think the original description that Andrew gave concerning this is a bit confusing; he mentions both a singleton (default) and instances (custom). I took this to mean one static instance as default, while more custom instances could be created if needed.

Concerning TextureFontAtlas, is there any reason why this atlas is specific to fonts? Why not TextureAtlas, which could be used by TextureFont and others? Come to think of it, is this what TextureCache is for? I've never used it or seen it used, only noticed it in the bottom of Texture.h.

cheers,

Rich

Re: TextEngine brainstorm

8 years ago

I've been once again reading boost's docs about shared_ptr's and I think I found the design pattern you used. It's nice, less code than a pimpl and as Dan says, easier to follow.

Got FreeType working on Mac with loading a ttf resource from the bundle, which uses the TextEngine's measure and measureGlyphs methods (had to switch them in TextureFont). It seems to be aligned with the baseline at the moment, but other than that looks really good:

I've pushed some of my ideas about the API for using the TextEngine to my repo here in hopes of stirring up some more conversation on the design. The major changes are:

Not a singleton. There is a shared instance, but more can be created for special circumstances. The type of shared instance is set through a static method, before it is first used.
FontRef's are acquired through a static Font::getRef method, which takes an optional TextEngineRef. This makes it so client code does not have to know about the TextEngine unless it has special needs.
TextEngine contains a font map to cache fonts of a specific type. The createFont methods are protected and there is a new getFont method that first looks in this map.

So here, you'd simply get a font by calling:

mFont = Font::getFont( "Helvetica", 20 );

If you wanted to use FreeType, you add this to the beginning of your App's setup() method:

TextEngine::setDefaultType( TextEngine::FREETYPE );

And if you want to use your own TextEngine, you do:

mTextEngine = TextEngine::create( TextEngine::FREETYPE );

mFont = Font::getFont( "Helvetica", 20, mTextEngine );

I think this is along the lines of what Andrew originally suggested. One nice thing about this design is that you can pass a custom subclass of TextEngine as the third parameter to getFont, if for some reason you wanted to do that. Some more details need to be addressed, like how does a Font implementation call to the right TextEngine. But so far, I can't see a reason why it should in any case but the constructor or getting the default font.

Re: TextEngine brainstorm

8 years ago

Cool, good to see it in action!

TextureFontAtlas is specific to fonts, it needs to know how to map a glyph area to a rect from a series of textures. This is specialized behaviour. Any Texture atlas implementation requires some concept of a subrect of another Texture, and Cinder does not have a representative class for this. But it's very easy to write your own Atlas class, the heavy lifting is done by BinPack and the Surface/Texture classes anyway.

Baseline issue: there's a difference in the way drawStringWrapped is implemented with Cocoa vs Freetype that needs fixing, see TextureFont::drawStringWrapped.

I've now changed TextEngine from a singleton and used dependency injection to associate a font with a TextEngine (required in places, grep for mTextEngine and see Cairo.cpp). Not sure if it makes sense to keep the Font::getDefault() and Font::getNames() methods around, but replacing them requires a bit of thought because of how they're called.

Caching the fonts in TextEngine sounds like a good idea. I feel the caching could live behind the create methods instead of getFont. But the engine should cache fonts created via DataSource too. The freetype backend doesn't implement a mapping from font name to font data - these are platform details handled by a system font manager. A simple way would be to create a unique map string key based on the font name/size or datasource path/size, depending on the constructor used.

sansumbrella

Re: TextEngine brainstorm

8 years ago

This is great work.

A note on positioning from the baseline versus the "top" of the letterforms:

Please keep type baseline-positioned. If we wanted to vertically align two pieces of text that use different typefaces/weights/sizes, we would need to be able to position them using their baseline. Please keep the text positioning coming from the baseline. For the textbox, we obviously want the text to fit inside the box (not on top), but for drawing individual lines of text the baseline stuff is important.

Some text Aligned with different text (same y-position, different typeface and size)

Re: TextEngine brainstorm

8 years ago

Baseline issue: there's a difference in the way drawStringWrapped is implemented with Cocoa vs Freetype that needs fixing, see TextureFont::drawStringWrapped.

The screenshot I posted above is from FreeType on OS X, by calling to the TextEngine for glyph measures in TextureFont.

Not sure if it makes sense to keep the Font::getDefault() and Font::getNames() methods around, but replacing them requires a bit of thought because of how they're called.

The Font::getDefault() method is convenient for classes that want to initiliaze there FontRef ivars without needed to know about TextEngine (it's used this way in a few different places). But maybe it should take an optional TextEngineRef parameter. I think getFontNames only belongs in the TextEngine.

Any Texture atlas implementation requires some concept of a subrect of another Texture

Yea I was wondering if it would be more useful if there was just a TextureAtlas class, and then TextureFont / TextEngine provided it with the subrects that it needed. I haven't had a chance to really look at your implementation, so I may just be blowing hot hair. I hope to look at it soon.

Re: Re: TextEngine brainstorm

8 years ago

The Font::getDefault() method is convenient for classes that want to initiliaze there FontRef ivars without needed to know about TextEngine (it's used this way in a few different places). But maybe it should take an optional TextEngineRef parameter.

The issue is that getDefault is a static member, while TextEngine is (potentially) no longer a static instance. Adding a TextEngineRef parameter to Font::getDefault doesn't solve the problem of how to obtain the appropriate TextEngine without a font instance, which is how getDefault is presently used. Once you have the appropriate textengine calling getDefaultFont is easy.

Re: TextEngine brainstorm

8 years ago

Re: caching fonts (I was having problems with those fancy quotes above... )

The way I handled the font cache was have a custom custom Compare functor:

(in TextEngine's protected interface )

typedef std::pair<std::string, int> FontType;
struct FontCompare
{
bool operator() ( const FontType &type1, const FontType &type2 )
{
return ( (type1.second < type2.second) || (type1.first < type2.first) );
}
};
std::map<FontType, FontRef, FontCompare> mFontMap;

I think this is a bit cheaper than appending the size onto the string, since you don't need to construct alot of otherwise useless strings with a boost formatter. Many times, it'll just be an int comparison that will fail first.

Re: Re: TextEngine brainstorm

8 years ago

In the greater scheme of things the time taken to format a few strings is insignificant, I wouldn't worry about micro optimizing it. But I do think it would be worth implementing it this way.

To handle DataSource identifiers requires extending FontType, defining FontType as a [ URL | filepath | hint | name, path, (float) size ] tuple/struct instead of using formatted strings to form the key. It adds a little complexity but has the advantage that the font size is an exact float comparison match.

Re: TextEngine brainstorm

8 years ago

From TextureFont.cpp:

void TextureFont::drawStringWrapped( const std::string &str, const Rectf &fitRect, const Vec2f &offset, const DrawOptions &options )
{
TextBox tbox = TextBox().font( mFont ).text( str ).size( fitRect.getWidth(), fitRect.getHeight() ).ligate( options.getLigate() );
vector<pair<uint16_t,Vec2f> > glyphMeasures = tbox.measureGlyphs();
// XXX track down the difference between these two calls
#if defined( CINDER_COCOA )
drawGlyphs( glyphMeasures, fitRect.getUpperLeft() + offset, options );
#else
drawGlyphs( glyphMeasures, fitRect, fitRect.getUpperLeft() + offset, options );
#endif
}

If you comment out the CINDER_COCOA branch the baseline should render correctly with Freetype. I'm guessing Freetype doesn't handle the drawGlyphs with baseline correctly, it's a TODO.

Re: TextEngine brainstorm

8 years ago

To handle DataSource identifiers requires extending FontType, defining FontType as a [ URL | filepath | hint | name, path, (float) size ]

Why won't a key of type pair< path | url, size > work? I know nothing about windows file handling, but I would think you'd be able to get the name of a font and use that. Is the hint necessary?

Also, if the user provided a font with the same name via the system, path, and url, they would all be stored as unique keys in the map. I think this is a good thing.

Re: TextEngine brainstorm

8 years ago

Because buffer data sources do not have a path, which is why a file path hint is necessary. Name collisions are possible between font name / font path / file path hint, so a type member is required to disambiguate the keys and preserve uniqueness.

Re: TextEngine brainstorm

8 years ago

I wonder if the font caching in TextEngine is really necessary?

The benefit is that it prevents multiple instantiations of the same font data from being created, but this is easy to avoid by the caller. A side effect is that shared font references have to be manually released from the cache by the TextEngine to be dropped. Dropping all your references to a font is not enough to free it forever, which is unexpected.

I'm starting to feel that caching is better implemented client side. Maybe there are use cases I haven't thought of?

Re: Re: TextEngine brainstorm

8 years ago

I've moved the font caching stuff to its own FontCache class now with getFont methods. I like having the option of using it without being forced to.

Re: TextEngine brainstorm

8 years ago

Hm, when you load a file on windows with the .rc thing, you don't get any file name through the resulting DataSource? That's weird to me, but whatever, all you need is a unique string for the first param, and then size for the second. Maybe it takes converting the ID to string.

About whether it's necessary to cache fonts, I'd like to hear what others think but here's my opinion: it's a perk that I'd want at some level of abstraction, and TextEngine looks like a nice place for it since it is a proxy for all font loading. There's no reason to keep creating the same font over and over again, and most apps only use a handful of fonts. But, many times there are many different components that need those fonts, meaning that somewhere they need a new shared resource. It could also be potentially expensive, if the font is constantly loaded from disk.

Baseline handling

8 years ago

I double checked the baseline handling and verified it's working consistently between Cocoa and Freetype.

The confusing part is the upper left corner of the rect passed into drawStringWrapped is actually used as the baseline for the text, which seems to contradict the comment doc. In the texture font sample the bounds rect is offset by the font ascent to maintain a uniform border around the text, so perhaps this is intended behaviour.

I *think* it should be calling the other form of drawGlyphs, passing in the fit rect and a clipping rect instead. But I'm not sure so have kept it as is for now.

Re: TextEngine brainstorm

7 years ago

Sorry to wake-up such an old thread but I guess it's the best place to ask the following questions:

1) Is it ok to assume that the TextEngine system described in this thread has not yet been integrated to any non-android branch of Cinder and that the most updated version is part of safetydank's android-dev branch?

2) I'm planing to use TextEngineFreeType on OSX (prototyping platform) and iOS / Android (target platforms): is it mature enough to work equally on all platforms?

Thanks!

Re: TextEngine brainstorm

7 years ago

The pull request issue is still open here:

https://github.com/cinder/Cinder/pull/165

I haven't heard anything since then. I keep it up to date in the android-dev branch for Android and Windows, but haven't built it for OS X yet. I use a Mac at work so I'm keen to get android-dev running on OS X soon, probably a week or so away.

Re: TextEngine brainstorm

7 years ago

Unfortunately in the perpetual triage of Cinder development, Text is currently sitting at the top of 0.8.7. Part of the reason is that the GL rewrite in 0.8.6 will affect it substantially. I intend to revisit @safetydank's work when we undergo the rewrite, but in the meantime if he's maintaining it the associated PR could possibly be applied to your own Cinder fork...

-Andrew

zsolt_ero

Re: TextEngine brainstorm

7 years ago

I just found this in a tweet today, might be interested for text rendering in the future, Python code is included in the paper:

Higher Quality 2D Text Rendering - http://graphics.cs.williams.edu/jcgt/published/0002/01/04/

Re: TextEngine brainstorm

7 years ago

The linked paper is by the author of freetype-gl and describes that library. I've used the same techniques in the freetype rendering code in the Freetype TextEngine classes, and credited freetype-gl in the Cinder-Android README.

© 2020 ZOHO Corp.
All Rights Reserved.
Privacy
Terms & Conditions
RSS
Powered by