Cinder-Protocol

in CinderBlocks • 5 years ago

https://github.com/BanTheRewind/Cinder-Protocol

With Cinder-Asio reaching a decent level of maturity, we're ready to move over to the protocol layer. If you've been following this thread, you'll know we've been working towards creating a comprehensive, high performance, and flexible networking library that is easy to use.

A big decision made along the way was to avoid the pitfall of most of these libraries and decouple the transport (how data is sent), protocol (what data is sent), and encryption layers. This should give us a solid toolkit to do anything from talk to hardware at high speed on a private network to implementing WebSockets from the ground up (and not deal with the mess that is WebSocketsPP).

WHAT IT IS

Cinder-Protocol follows a similar philosophy and style as Cinder-Asio in that there is a lot of inheritance. This makes it easy to implement protocols that follow the top line + header fields + body format (HTTP, FTP, SMTP, WebSockets, etc) while allowing the flexibility to work with non-standardized fields, or even invent your own protocol.

Right now it, looks like this:

ProtocolInterface

HttpInterface

HttpRequest
HttpResponse

The base class is really generic, just supporting the concept of a key-value pair header and a separate body, alongside some utilities for converting between strings, byte buffers, and a << for just streaming the entire object. HttpInterface adds HTTP versioning to to base class, while the HttpRequestand HttpResponse classes finish out structuring the data (ie, status code, reason, etc).

The block includes sample projects for a web client and HTTP client and server. The master branch has an in-progress FTP sample. It works, I just need to write out more communication and find a set up a test server.

WHAT IT ISN'T

Cinder-Protocol is not meant to support standards. That is, you can create and parse a valid HTTP request and/or response, but it is going to remain agnostic about what the content of your header fields and body are. The body is just a byte buffer. The header fields are just pairs of strings. If a web site has a "non-standard" field, Cinder-Protocol will support it without any sort of warning. It's up to you to filter that stuff out.

This library focuses on web protocols. OSC is considered a "protocol", but doesn't fit the definition of "protocol" the same way HTTP, FTP, or SMTP do.

Side note, OSC is more akin to JSON or XML in our eyes. Mike Latzoni has been working on an OscTree not unlike XmlTree or JsonTree. It's close, and it's totally awesome...

https://github.com/djmike/OscDev

HELP

We'd love PRs on this one. If you have lots of experience implementing SMTP, FTP, RTMP, etc, we'd really like to see versions of those akin to the HTTP classes. FTP has already been started.

Anywho, give it a shot and leave some feedback. If there are any actual issues, please bring them up in the repo, not here.

http://www.BanTheRewind.com

Replies(5)

pushpop

Re: Cinder-Protocol

5 years ago

Hey this looks really great man!

Will undoubtedly come in handy early and often.

Only thing is, do you have to create a session separately in order to make an HttpRequest? Maybe I'm missing a way to make it simpler.

I recently made an HttpRequest class that managed it's own session so you could have an interface something like this:

HttpRequest::Create("serverAddr", "Port")->beginGet("/get/endpt/", [](std::string response){
Console() << response;

});

And everything gets cleaned up after.

Maybe that's a dirty hack, but It's useful. :p

Otherwise, thanks for getting this all in! Can't wait to have this all in one place.

bantherew..

Re: Cinder-Protocol

5 years ago

The session is part of the transport layer while the request is in the protocol layer. These are intentionally decoupled. You can use Cinder-Protocol with Cinder-Asio, with boost::asio (now becoming asio::), Winsock, POSIX, anything. And you can use Cinder-Asio to send any kind of data you want (RTMP, HTTP, OSC, whatever).

In fact, these two blocks were born out of the frustration of having to use libraries that keep these layers coupled. OSC, for example, seems to always show up in the form of a library which includes a transport layer that I either don't want to use, conflicts with something else I'm using, or doesn't work with my environment. OSC is just a way of formatting data and should have nothing to do with the mechanisms that move it.

I have a couple of projects that have HttpClient and HttpServer classes which combine the two. They're basically the sample applications broken out to a reusable class. These both come with extensions of the event handler classes which are super useful for making multi-threaded servers, etc.

http://www.BanTheRewind.com

pushpop

Re: Cinder-Protocol

5 years ago

Yeah totally understand (and appreciate) the decoupling.

Just wasn't sure if there was a quick way to (re)couple the two.

Makes sense to just make a helper to do the dirty work.

kitschpat..

Re: Cinder-Protocol

4 years ago

Hey, thanks so much for this block.

I know I'm raising dead threads here, but wanted to add a comment in case anyone comes around wondering about re-coupling these as pushpop did.

I recently wrote a small wrapper on top of Cinder-Protocol with this use-case in mind... I actually didn't see pushpop's similar approach until after the fact. Anyway here's where I landed:

easyHttp->request("http://example.com",
[](std::string successResponse) {
CI_LOG_V("Success!" << successResponse);
},
[](std::string failureResponse) {
CI_LOG_V("Fail!" << failureResponse);
});

There are overloads of the same method that let you pass HttpRequest objects and get an HttpResponse through the callback for more elaborate transactions.

Multiple requests are managed automatically.

Just putting this out there.

This currently lives in my "catch-all" repo of assorted junk. Here are the relevant files:

Re: Cinder-Protocol

4 years ago

In practice, I often end out making something like this. But I do still like to leave the two sides decoupled so I can use one instance of a TcpClient that can do everything from image downloads to local network communication to email. Depends on the needs of the app.

I would recommend adding a way for the user to access the HttpRequest on error. I usually create a HttpRequest queue of anything that needs to go out, then quickly work through it as connections become available. If the request fails, it's really handy to be able to access the HttpRequest at the time of the error and push it back into the queue to try again.

This thread was about to become un-dead soon, actually. If you've been following along, you may have noticed I've started a "ssl" branch in Cinder-Asio.

https://github.com/bantherewind/Cinder-Asio/tree/ssl

It's close to finished (just needs some certificate verification cleanup and a sample app) and I've already battle tested a version in production. This is necessary for most modern API communication, and should open the door for a super lightweight, low-level WebSocket system in Cinder-Protocol. While I've used WebSocket++ with success, it's a very large dependency for what is usually a tiny part of a project, and one which takes work to not clash with other code since it couples the protocol and transport layers.

This requires OpenSSL. Still trying to weigh whether to include static libs or not. It's a pain for most users to build, but it's also important to be able to let users quickly update the lib if any critical security patches arise. Hmmm...

http://www.BanTheRewind.com

Top Reply

Response title

Attachments

Cinder-Protocol

Replies(5)

Re: Cinder-Protocol

Re: Cinder-Protocol

Re: Cinder-Protocol

Re: Cinder-Protocol

Re: Cinder-Protocol

Statistics

Tags

Actions

Related Posts