Articles(Just tidying up some loose ends.)
I once complained about Python Logging and its performance for real-time systems.
It was suggested in the comments by the owner of the Python Logging module that I make a custom version for my needs. I protested that I didn’t think I was that special, and it should be available to everyone.
Nothing happened for a few weeks, until I got bitten by my inaction and I decided I should try to make a solution for everyone. In that same post, I laid out the design for a solution that should work for everyone.
The next day, I pointed out a difficult problem with Python shutdown which sounded like it should be a lot easier than it was.
One of the OddThinking regulars, configurator, pointed out that I was being too thread-averse. Being willing to create threads more frequently gave me a simple solution. Performance tests showed I need not be that worried about the cost of threads in the typical case. Thanks, configurator. I proceeded with the implementation.
I have shared that solution with others, and it has earned a few raised eyebrows. However, configurator’s suggestion seems to be the only one that gives correct behaviour, which counts for an awful lot. It also, in the typical case, is quite performant.
The concern is that it may perform badly in the case of a run-away thread emitting log messages in an infinite loop. Without the plain logging solution, this would be I/O bound, sending emails as quickly as it could, while the rest of the system kept performing normally. Under my implementation, I would expect the number of threads to explode. I assume this would have a heavy toll on other threads and processes in the operating system.
So I am still open to less thread-intensive implementations, I just don’t know what they are. Please revisit the discussion if you have any ideas.
Anyway, I completed the implementation called the Nonblocking Log Handler, made it available in PyPI, talked it up at a Python User’s Group, and I have been using it in anger for 2-3 months.
Today, I finally got around to the online documentation.
So, I can now pat myself on the back by contributing another one-person dead-end project to the Open Source Community. If just one other person finds it useful… it will have two more users than most of the software I have written in my career 🙁
Your feedback, as always, is very welcome.
Comment by configurator on August 25, 2010
I thought my suggestion would only create a thread while there isn’t one already alive? How would you have an exploding number of threads, then? You’d just have the one thread, busy in a pop-from-queue-and-send-email loop.
Comment by configurator on August 25, 2010
Looking at the source code I see there is only one consumer thread that will ever get started. Maybe I missed something, but I fail to see the thread explosion.
Comment by Julian on August 25, 2010
configurator,
I think it is best if I continue this discussion on the post related to the shut-down issue.
Comment by Julian on August 27, 2010
What I learnt last night about Python packaging.
easy_install is a linguistically-weird form of double-negative. As simpler form would be “tricky_stall”.
Follow copy-and-pasted recipes from online tutorials when initially writing your setup scripts… except when the recipes turn out to be wrong, months later, causing weird symptoms, such as command-line options (e.g. bdist_inst) to disappear.
Using “dry-run” mode when installing is a great way to find out about errors before you damage your install, like the weird complaint that a zip-file isn’t in a zipfile format. It turns out that error means “You are running in dry-run mode; this works in a real install.”
PyCheesecake is an automatic style-checker for your installs, which helps you to understand what is missing. It is in “very active development” according to its homepage which means there’s been no sign of anyone working on it for 3+ years. It is also broken.
There is a formal way to add “thanks” credits to a package. If anyone can tell me how, they will receive formal thanks.
Setup tools allows you to easily upload several different variants to meet your customer’s needs – like Ubuntu’s easy_install, that will gladly pick its favourite format – the Windows installer – and then fail because it isn’t an appropriate format. If you don’t upload the Windows installer, it will settle for what it can get – such as the platform-independent egg file – and succeed.
SetupTools let you specify your “manifest” with wildcards, which is a way of listing all the critical files you need to include in the distribution. It turns that into a list of complete filenames. Which is nice, because then you know the names of all the critical files it hasn’t actually included in the distribution.
There is this magical place, where a failed install of the logging functionality, causes your server code to fail – but without logging a message because the logging functionality has failed. But then your watchdog restarts the stopped process. So when you check your error log there is nothing. When you check the process is running, there it is – always in the process of falling over. But – and this is the sweet part – this is all consuming massive amounts of CPU, which cause another watchdog to email you saying there is a performance problem. When you run “top”, you can see the load is 10x higher than normal, but there is nothing consuming much CPU – all the CPU-chewing processes die, somehow before they can get a ranking as one of the top processes.
Comment by Julian on August 27, 2010
The above comment is a long-winded way of saying “There is a new version up, but there is no Windows installer, and it is still missing some non-mission-critical items, like thanks to configurator.”
Comment by John Y. on September 1, 2010
Re: Python packaging
All I can offer you is sympathy. This is a well-known and much-lamented problem in the Python community (and probably many other open source communities, including Linux itself). There is someone working very hard on revamping the distribution utilities in Python’s standard library, but I must admit I’m not optimistic that it will be such a huge improvement that everyone will flock to it and repackage their existing software with it, especially after working so hard on their existing (hellishly difficult) packaging. Which means that, while some new projects will use the new facilities, that only means there will be Yet Another Way to package Python software. It seems this problem may be inherently Perlish in its “more-than-one-way-to-do-it”ness.
Personally, I prefer packages which “install” by way of simply being copied into site-packages. (Is this what an egg is? I have never really understood.) I suspect this becomes less attractive the larger and more complex the package being distributed, or the larger and more complex the third-party ecosystem which is already installed. But for something small, and implemented in pure Python, I am happiest (least grumpy, and least fearful of messing stuff up) just dropping it into site-packages myself, thankyouverymuch.
Comment by John Y. on September 1, 2010
Re: Your nonblocking log handler
Congratulations on releasing this! You and configurator deserve a lot of credit. Maybe the rock-star ninja programmers out there will disagree with me, but I think what you’ve done is highly nontrivial. (And I certainly encourage them to present better solutions if they can come up with any.)
Comment by Aristotle Pagaltzis on September 1, 2010
And yet, funnily enough, it’s Perl that has the CPAN and largely One Way To Do That. :-)
(The latter is a bit of a lie, but the landscape is nowhere near as heterogeneous as it apparently is in Python. (I’ve been hearing Pythonistas grouse about the situation for as long as I can remember.))