DotCloud releases support for Perl, aka Camel-as-a-Service
blog.dotcloud.com
blog.dotcloud.com
I would love to know more about the discussions you had with miyagawa, and how the design for this evolved. Anything this elegant must have taken a ton of iterations.
Oh, and may I have an invite, please?
http://www.dotcloud.com/account/create
A5FxUv 0HJldL
oM0PPT l6CWNG
4iP5YR ggCJ1k
mSMOaV h1JvT7
rh4aoT BkeCkR
2xYfBf uyR3d4
g2ZBP9 hJ0JKc
OWx5qu cmf9z9
bAlvX4 W6yt3q
Wi42Ft XgFmfP
nnvVxf kfHuMh
kJjg4Q eXV7Wn
Djxozb chSFmG
gt3nQb TNNupl
oB2JNm OgDsHW
thEAaq oZx5mv
I1pMjR nWeKhz
vjd19G YiNr1O
wG2Uqm XBQp3P
wNZJAE yRfnON
8tiohx AIg6BH
W2IU3h xmfjZ5
E5Ttq5 GWlpMK
x8lUjd Vfbjxi
MEDa7H HnytsUThat list must have been exhausted unbelievably fast, I don't think even Linode VPS giveaway a few months back was that successful.
DotCloud++ Miyagawa++ Perl++
Good luck !
Where is DotCloud hosted?
DotCloud runs on Amazon EC2. More specifically, we run on the us-east-1 region, across multiple availability zones.
http://www.modernperlbooks.com/mt/2011/03/why-psgiplack-matt...
It resumes quite well my opinion and concern regarding the doc.
- No Apache????
- why force a rather devish level PSGI module on all web applications???? oh, yeah, you've been advised by the guy who made the port...
i've been using CGI::Fast under Apache for years and it has been working just PERFECTLY fine.
Do not see the need to change especially when
the PSGI doc is so badly written...
At least the doc states that people who supports mod_perl/CGI::Fast will not see much benefit of switching to PSGI...
So far all alpha testers seemed very happy :)
To tell the truth, this could be a good occasion to try to make the in-house app works in such environment but
1- don't see any real benefit
2- don't want to decipher the doc
3- no time (will have to work extra hours)
you should definitely support both environment and not base your choice on the sole feedback of one person (you know... TMTOWTDI)
Well, actually, we didn't pickup Plack from its author. When people started asking about Perl, we just replied with another question: "which framework? how? what?", and we tried extract the most common denominator.
The vast majority told us about Mojolicious, Dancer and Catalyst. There seemed to be a pattern :-)
Regarding the server interface, a lot of people seems to agree over Plack. We were the ones asking "hey, what about FastCGI?" - since I personally never had met Plack before. We were shown the Plack::*::FCGI modules and thought "okay, that seems cool".
Then we had a look at the server options. We had already used UWSGI for the Python stack, and UWSGI has a PSGI module. Some might say that it's a bit experimental, but the authors are awesome people, always willing to fix bugs as fast and efficiently as possible.
Of course, the author of Plack helped us to get everything straight - but paradoxally, he was also the one pointing us to cpanm, perlbrew and basic stuff like Makefile.PL.
Now, to get back to you: what does your app look like? We would like to help and see how we can get it to run on DotCloud!
>Some might say that it's a bit experimental,
Because it is.
It's nice to play with it though. But not quite yet to use for our clients.
> but the authors are awesome people, always willing to fix bugs as fast and efficiently as possible.
Won't be able to say to my client: well, you know, we've tried this experimental stuff but we met these bugs we can't do anything about.
"hey, what about FastCGI?"
This was a very good question and I would have answered go for it... Apache+FastCGI+CGI::Fast. That's quite common nowadays and I think well understood, well documented.
cpanm is not necessary, I've never used it and am happy with cpan. oh, right, cpanm is by the same author right?
perlbrew&locale::lib are nice things to have though.
> We would like to help and see how we can get it to run on DotCloud!
Apache+FastCGI+CGI::Fast,that will do for me.
Apache+mod_perl,that will do (even though i don't use it).
Having at the cutting edge technologies is nice, don't get me wrong but it would help to have at least a more stable or at least bullet-proof environment too.
I'm REALLY looking forward to playing with DotCloud.
Plack/PSGI is not "cutting edge". It's the way to do it now, not tomorrow. You do Miyagawa a disservice implying he's pumping his own code.
PSGI stable, sane and the way you want to be doing things.
Indeed if you write an app to run on DotCloud; you can also run the same app on your laptop, Phenona, any other server -- etc etc.
As someone else said (more politely), you'd have to be nuts to do an HTTP app now and not take advantage of it.
In your company or in general? Because if it's in general, I would be more than happy to see where you get your stats.
> You do Miyagawa a disservice implying he's pumping his own code.
Well, I do think that a not even 2yo spec,a set of modules that are 6~18 months old pushed on a cloud service like that with no back up plan for more "traditional" systems is a way to push its own system. we can agree to disagree on that. which percentage of providers offer plack with nginx nowadays compared to Apache with FactCGI?
Any time anybody reports a problem with them, we go out of our way to solve it - avoiding wheel reinvention is important to us to speed development but we understand that using a module makes us responsible for it installing for end users.
if you really want CGI.pm, there are Plack modules to convert a PSGI environment to a CGI environment - descended roughly from the Catalyst::Controller::WrapCGI that Opsview sponsored Shadowcat to write years ago.
And remember, PSGI might appear to be new but much of the guts of Plack are pulled from previous framework-specific code so the various workarounds for server insanities are already there. It's a lot more reliable than you appear to think.
I see. Perhaps my vision of the state of PSGI is not right indeed. Nethertheless, my apps run under Fast::CGI and I really don't see the advantage of PSGI.
PSGI has been developed in 2009, then nothing for a long time (and as it hasn't spread as wild fire across providers i don't think there is enough real dev around it to assure that it is "stable" enough), now, a dev release, sure... the doc is unclear as to what it brings on the table, why it should be used, how to be used, the improvement it brings if any are really not clear... "you will be glad in the future..."...yeah, i need more than that to get people around me starting to use it.
FastCGI has been working for a long time, clear, easy and already "cross-server"...
All in all, the doc is a real problem and no hard evidence of it being used is, to some extend, an other.
Nobody in their right mind has been using CGI::Fast directly for years - we've done it via adapters through e.g. Catalyst::Engine::FCGI for Catalyst and equivalents for other frameworks.
PSGI is just a means to present various HTTP bearing things to the app - in fact the FCGI.pm maintainers all now deploy their applications via Plack::Handler::FCGI.
All perl frameworks of note now have support for PSGI, and Catalyst is dumping its engine modules in favour of Catalyst::Engine::PSGI - we've been heavily involved in improving the Plack ecosystem to ensure that'll be a sane choice.
There's plenty of assurance that it's stable enough.
Go DotCloud for forward thinking choices here.
Could you extend on that? Been using it for years,just works fine.
>PSGI is just a means to present various HTTP bearing things to the app
I really still don't get the benefit though. But perhaps I should dig into it more (for now, the python implementation doc was helpful but still did not bring me that much clues as to the real benefits about it)
I assume you're looking at the non-dev PSGI specification on CPAN. We're working on revising the documentation to upgrade the specification to 1.1, as well as improving the FAQ document.
Just to address your concern, I improved the part where it explains the benefits of switching to PSGI: https://github.com/miyagawa/psgi-specs/commit/e3e756cf1eebe8...
Also, for more generic overview and introduction of what PSGI is and why it is useful, take a look at my slides at OSCON 2010: http://www.slideshare.net/miyagawa/plack-at-oscon-2010
- miyagawa
> I assume you're looking at the non-dev PSGI specification
yes.
> PSGI specification on CPAN
Not really sure it is a good idea to put a spec on CPAN(I know you can find books on CPAN but I really do not think this is the right place for such matters but especially when PSGI/Plack confusion certainly exists.well, that just my view of things)...
Regarding your additions:
>You can stop writing code to support many web server environments.
As your slide shows, CGI.pm handles quite well many of the environment you list. It has been in development for more than 10years, updated very regularly to fix bugs, even browser based one, the doc is easy to read. The perl based server you list were built for PSGI in mind so you sell me some candies and the toothbrush that goes with it.
> Your framework can now use all of Plack middleware components.
session, caching,etc. All of these "middleware" already exists since a very long time. so it is just yet an other addition,an other way of doing it. Some of these PSGI middleware were extracted from Catalyst. Catalyst authors decided to create their own set of engine instead of choosing to use CGI.pm (unlike CGI::Application/Jifty building upon it), linking to your "gross." conclusion- in your slide.
> You can test the application using the consistent L<Plack::Test> interface.
WWW::Mechanize works like a charm for such matters. (perhaps your Plack::Test differs though, did not look into it so this might be a valid point)
Regarding your slide:
The diagram you show is at best simplistic, at worst wrong (factcgi->lighttpd). It deviates from reality to get to your point.
I find the "CGI.pm? meh" slide very disrespectful and all by itself shows the ambient snobbish attitude towards this very useful module(inherited from the Catalyst community mindset?)
so all in all, I still don't get the problem it is trying to solve(stealing from python you wrote. an other language, an other culture, an other context) I still don't get the benefit out of it and why I should use it.
I think I am a lost cause^^; I am sorry. It's not that I am against new things but here, I really don't get the advantage of this spec and everything that goes around. I have a bootstrap package that launches the FastCGI run loop then delegates to modules that do there job and print the resulting body at the very end (it's a MVC like framework,nothing fancy) so I don't think it would hurt much to add support for this but I won't do so as long as I don't get a real + in doing so.
Using Nginx is not much of a matter. (Nginx+FastCGI should be fine)
They are also available on github with POD markup, with the full version controlled history: https://github.com/miyagawa/psgi-specs
> CGI.pm handles quite well many of the environment you list. It has been in development for more than 10years, updated very regularly to fix bugs, even browser based one, the doc is easy to read.
Yes, that's why I collaborated with CGI.pm authors and maintainers (Lincoln Stein and Mark Stosberg) from day 1 - like the possibility to include PSGI support directly into CGI.pm, which ended up in an extension module i co-authored with CGI.pm maintainers and put up on CPAN as CGI::PSGI, the CGI.pm compatible interface. T
There's also CGI::Emulate::PSGI which allows to write any CGI scripts unmodified under the PSGI environment.
> session, caching,etc. All of these "middleware" already exists since a very long time. so it is just yet an other addition,an other way of doing it.
It's not just an "addition" - if there are N frameworks, the caching modules need to be written and maintained for N frameworks - with PSGI unification, it should be written once and tested and maintained once - that way we can avoid duplicate efforts to fix bugs and make improvements, etc.
> Catalyst authors decided to create their own set of engine instead of choosing to use CGI.pm (unlike CGI::Application/Jifty building upon it)
and the upcoming Catalyst 5.9 removes all engines in favor of PSGI/Plack interface, like Jifty did a while ago.
> WWW::Mechanize works like a charm for such matters. (perhaps your Plack::Test differs though, did not look into it so this might be a valid point)
That's why there's Test::WWW::Mechanize::PSGI exists - you can run the exactly same test against a live server as well as in-process testing through the PSGI interface.
Great. perhaps getting rid of it on CPAN could be nice.
> It's not just an "addition" - if there are N frameworks, the caching modules need to be written and maintained for N frameworks
I can not agree with that point. You are almost forgetting CPAN here-which is funny coming from you- (The strength of Perl that many languages do not even dream to have)... The actual modules were context agnostic and it's not because you have N frameworks that you MUST have N xxx(caching,session) frameworks. It's just that some framework authors want to reinvent the wheel.no more, no less(in other languages, without an infrastructure like CPAN this can happen more than it ought to, indeed!).
so the logic: N web framework = N xxx sub frameworks is just biased. It has always been possible to not duplicate efforts, avoid bugs with different code trying to solve the same problem. It's just that again, some framework did reinvent the wheel where they should have been using what already works (or help on it).
> That's why there's Test::WWW::Mechanize::PSGI exists
you are just telling me that there is yet an other way of doing the same thing but in the PSGI environment.
All in all, I still don't see what the PSGI spec/protocol and the modules that evolute around bring to the environment. I don't see why we should have a superset of CGI,which is a language agnostic protocol,that binds with Perl via PSGI. Really don't get the idea...Is this binding between Perl and the protocol lead to way much better performance?(better than FastCGI?)
I've never used mod_perl as it is only useful for Apache but I knew that using FastCGI could open me many servers and better performance without having to change much and care about even the language.
I guess my environment is too far from the problems you seem to encounter and fix.
They are new things where I think, yes, this is going to be good but here, the "pintokita" moment does not arrive at all.
I can only hope that DotCloud will offer a simple support for fastCGI (CGI::Fast) with Nginx.
By the way, adding some script examples for CGI::PSGI and other related modules would be much appreciated (The doc is too elliptic and assume we know already that much about the PSGI context and Plack system)
Thank you for your time and efforts anyhow!
The SYNOPSIS has the complete working example code that you can run with plackup, or any PSGI compatible web server - there's no much need for detailed docs about the interface since you don't need to change anything, other than actually being documented.
That said, a doc patch is always welcome - on github fork or via RT.
For the last chance, I'd suggest you more links where people explain the benefit of PSGI over CGI, Apache, FastCGI, HTTP::Engine or anything - in case you haven't checked them out. If you still don't get it and think Apache + FastCGI + CGI::Fast is the best thing in the world and you absolutely need nothing else, then I'm sorry, but that's fine.
http://www.perl.org/about/whitepapers/perl-plack.html http://www.simon-cozens.org/content/i-finally-get-psgi-and-p... http://blog.patspam.com/2009/plebgui-webgui-meets-plack
I would like to but I need to understand how this works to provide any patch whatsoever. What about porting a CGI::Fast application into the PSGI stack then?
while (my $req = CGI::Fast->new) { myApp->run($req); }
Is there a handler/wrapper for that?
I've read all the articles but it did not help much. WebGUI discovered FastCGI via a PSGI implementation but it's FastCGI who brought the speed, not PSGI per se. So it brings me back to my point...
> think Apache + FastCGI + CGI::Fast is the best thing in the world and you absolutely need nothing else
I think FastCGI was there to solve indeed a problem and did quite well at doing so. It could be Apache/Nginx/IIS + FastCGI&CGI::Fast, it would work too (Apache/IIS is from experience but Nginx is just an educated guess).
I've decided a long time ago that I will never go the mod_perl way (or at least, will not use their low level access API) then switched to FastCGI and been happy since then.
I still don't get it but perhaps one day it will hit me. I guess when somebody will bring a "Middleware" that will meet a need.
In the meantime, I will try to play with it on my spare time. I guess it will be the easiest way to discover what I might be missing...
If DotCloud does not offer this infrastructure(Nginx+FastCGI), it's ok, I still can use our actual providers and it will work (exactly the WebGUI article example for cheap providers!)
Thank you for your time!
Assuming that while loop is in your bootstrap FastCGI script, you can instead have an app.psgi (or whatever named) file with the content:
use CGI::PSGI;
my $app = sub { my $req = CGI::PSGI->new(shift); myApp->run_psgi($req) };
For most web frameworks, the change should be minimal and straightforward: for example, CGI::Application needed less than 10 lines of code to implement this. http://search.cpan.org/~markstos/CGI-Application-PSGI-1.00/
(I implemented the original code, and markstos, the maintainer of CGI.pm and CGIApp now took it over. As you can see there's a small hack to capture the output - they're working on removing this hack by implementing the PSGI natively inside the CGIApp codebase)
> WebGUI discovered FastCGI via a PSGI implementation but it's FastCGI who brought the speed,
No, they got a performance boost with our preforking standalone HTTP server, which is currently called Starman, not just FastCGI.
Speaking of FastCGI, although Plack has a FCGI.pm-based FastCGI handler, i've been working on another FastCGI based preforking PSGI server called fastpass. https://github.com/miyagawa/fastpass
It is XS dependency free (unlike FCGI.pm and CGI::Fast) and the performance is still the same with FCGI.pm, roughly like 4000 requests per second on my laptop, with a simple HelloWorld app via an nginx frontend. I guess we could do even better by doing optional XS parsing with pure perl fallback as well.
FWIW for a comparison, with Starman I get 7k and Feersum gets 9k requests per second on the same machine. (Of course the number is not that significant since in the real world, your application does more IOs, templating stuff and database handling, and the qps would be much smaller)
Again, the nice thing about all of these things is that your code, and everyone else's code, don't need any line of code change to support this new server, once you get PSGI.
> In the meantime, I will try to play with it on my spare time. I guess it will be the easiest way to discover what I might be missing...
I'm pretty sure you will :)
Yes it is.
> For most web frameworks, the change should be minimal and straightforward
Indeed, it seems so.I'll try it out.
>No, they got a performance boost with our preforking standalone HTTP server,
I was talking about these lines in the article, where Starman is not used at all:
--------
I’ve codenamed the project “PlebGUI“, which I think aptly describes the way it makes it possible for the little people to run WebGUI on low-cost shared hosting.
And it actually works. Take for instance plebgui.patspam.com, a demo PlebGUI site site running in FastCGI mode on HostMonster (the prototypical low-cost shared webhost).
---------
Even though, the application did have even better performance via Starman but it is more a server matter than a PSGI related matter.
>fastpass [...] is XS dependency free (unlike FCGI.pm and CGI::Fast) and the performance is still the same with FCGI.pm, roughly like 4000 requests per second on my laptop, with a simple HelloWorld app via an nginx frontend. I guess we could do even better by doing optional XS parsing with pure perl fallback as well.
THIS is interesting.
- Why send back an array ref instead of using the CGI way? What is the advantage it brings?
If I want my app to run in a simple FastCGI environment (with the CGI protocol), this will not be possible anymore so I will be dependent on the Plack suite or create 2 versions: PSCI enabled and not PSGI enabled (this will not be much but still, 2 is worst than 1).
- Are all the $ENV variables available? (SCRIPT_NAME for example?)
- All the plack Middleware sounds much like externalized Catalyst modules. I don't really see what advantages it will bring to framework like Mojolicious for example (they will certainly never ever used any Middleware stuff from the Plack namespace as this is at the heart of their policy so they will just get access to the servers that implements this spec instead of using the normal CGI protocol) If you put them all back together, you get Catalyst.
So if you start to build a new framework from scratch, yeah, this might be nice. Everything has been done, you just pick what you want but else...
If you design a fibonacci() function, would you make it print the result to STDOUT, or return the result as a return value?
If the specification is to print to STDOUT, a web server needs to make a trick to capture the output using tie, PerlIO or anything else, just like FCGI.pm does, and that's inefficient.
> - Are all the $ENV variables available? (SCRIPT_NAME for example?)
http://search.cpan.org/~miyagawa/PSGI-1.03/PSGI.pod#The_Envi...
> I don't really see what advantages it will bring to framework like Mojolicious for example (they will certainly never ever used any Middleware stuff from the Plack namespace as this is at the heart of their policy
See this post by the Mojolicious author, how to use Plack middleware for ANY Mojolicious based apps http://blog.kraih.com/mojolicious-and-plack
Also for Dancer: http://advent.perldancer.org/2010/22
I see. That's a valid point. So when I asked if by design the protocol would offer better performance, the answer is yes?
Is this inefficiency so bad that changing it to an array ref brings THAT much improvement though?
It was easy to change the output but I will have to create a thin wrapper around FastCGI/PSGI to make it transparent. something like this detection done in Mojo becomes necessary:
# PSGI (Plack only for now) return 'psgi' if defined $ENV{PLACK_ENV};
# CGI
return 'cgi' if defined $ENV{PATH_INFO} || defined $ENV{GATEWAY_INTERFACE};
I'll continue porting the app and see how it behaves.
Thank you for your time.