From daf73af79354bfe36d4532438aceacd2a866e362 Mon Sep 17 00:00:00 2001 From: David Runge Date: Sat, 20 Apr 2019 00:10:43 +0200 Subject: posts/*: Moving all posts to year-based ordering. --- ...-longevity-of-a-htc-one-s-using-cyanogenmod.rst | 427 ++++++ posts/2016/201604-linux-audio-conference-2016.rst | 376 +++++ posts/2016/201604-mssw.rst | 240 ++++ posts/2016/201609-darmstadt.rst | 430 ++++++ posts/2016/201609-letsencrypt.rst | 893 ++++++++++++ posts/2016/201610-uwsgi.rst | 1468 ++++++++++++++++++++ 6 files changed, 3834 insertions(+) create mode 100644 posts/2016/201604-extended-longevity-of-a-htc-one-s-using-cyanogenmod.rst create mode 100644 posts/2016/201604-linux-audio-conference-2016.rst create mode 100644 posts/2016/201604-mssw.rst create mode 100644 posts/2016/201609-darmstadt.rst create mode 100644 posts/2016/201609-letsencrypt.rst create mode 100644 posts/2016/201610-uwsgi.rst (limited to 'posts/2016') diff --git a/posts/2016/201604-extended-longevity-of-a-htc-one-s-using-cyanogenmod.rst b/posts/2016/201604-extended-longevity-of-a-htc-one-s-using-cyanogenmod.rst new file mode 100644 index 0000000..2d7531a --- /dev/null +++ b/posts/2016/201604-extended-longevity-of-a-htc-one-s-using-cyanogenmod.rst @@ -0,0 +1,427 @@ +.. title: Extended longevity of a HTC One S using Cyanogenmod +.. date: 2016-04-02 20:00 UTC+02:00 +.. modified: 2016-04-03 20:00 UTC+02:00 +.. tags: cyanogenmod, android, apps, f-droid, guardian project, tor, htc one s +.. category: mobile +.. slug: extended-longevity-of-a-htc-one-s-using-cyanogenmod +.. authors: David Runge +.. summary: Or howto setup Cyanogenmod on a HTC One S and live with F-Droid happily ever after + +The mobile +__________ +| I own a quite old - at least by today's standards of planned obsolescence for every consumer device - |htc_one_s| (2012), that won't be receiving any more Android upgrades (|htc_one_s_last_update|: version 4.0.4) or support by its manufacturer directly or any distributor for that matter. +| It's quite a nice, small and lightweight phone, that by now has seen the world and besides a self-administered chassis change hasn't yielded any serious problems. +| The phone doesn't have much RAM or much space. Neither does it have a microSD card slot, |abbr_nfc| or other fancy new stuff that people seem to need. +| + +CyanogenMod +___________ +| The alternative firmware for some mobile devices |cyanogenmod| is available for free and offers a community driven development branch of the Android kernel, without the Google and mobile vendor bloat. +| Have a look: Maybe your phone is amongst the |cyanogenmod_supported_devices|? +| Not all vendors allow flashing other firmwares though. Some prevent this completely (watch |28c3_doctorow| for an in-depth comment on this), others let you do it by voiding your warranty, yet others just let you do it, because they are awesome. +| All |cyanogenmod| capable devices have development codenames. For the |htc_one_s| this is "|cyanogenmod_ville|". +| Each device offers a |cyanogenmod_ville_release_channel| (with stable releases) and a |cyanogenmod_ville_development_channel| (with nightly builds for the adventurous and daring). +| An |cyanogenmod_ville_install| explains how to get |cyanogenmod| on your device. +| + +F-Droid +_______ +| |f-droid| is an alternative app store to |google_play|. It only offers |free_software|. This is pretty awesome. +| Some useful apps I like, use or have used: + +* |f-droid_afwall| - Control network traffic +* |f-droid_antennapod| - Advanced podcast manager and player +* |f-droid_ardroid| - Remote control for |ardour| +* |f-droid_cadroid| - Certificate importer +* |f-droid_connectbot| - |abbr_ssh| and local shell client +* |f-droid_conversations| - |abbr_xmpp| client +* |f-droid_davdroid| - Contacts and Calendar sync +* |f-droid_droidshows| - TV series browser and tracker +* |f-droid_f-droid| - Application manager +* |f-droid_firefox| - Web browser +* |f-droid_irssi_connectbot| - Specialised |abbr_ssh| Client +* |f-droid_k-9_mail| - Full-featured email client +* |f-droid_kontalk| - Community-driven messaging +* |f-droid_kore| - Remote control for |kodi| (XBMC) +* |f-droid_logical_defence| - Encyclopedia of logical fallacies +* |f-droid_mpdroid| - |mpd| (Music Player Daemon) client +* |f-droid_mupdf| - Lightweight document viewer +* |f-droid_owncloud| - Synchronization client +* |f-droid_owncloud_news| - News/feed reader +* |f-droid_owncloud_notes| - Client for ownCloud Notes App +* |f-droid_oandbackup| - Backup manager +* |f-droid_open_camera| - Camera App +* |f-droid_openkeychain| - Encrypt files and communications with OpenPGP +* |f-droid_opentasks| - Keep track of your list of goals +* |f-droid_openvpn_for_android| - |openvpn| without root +* |f-droid_openvpn_settings| - |abbr_vpn| settings +* |f-droid_osmand| - Offline/online maps and navigation +* |f-droid_orbot| - |tor_project| (anonymity) client +* |f-droid_orwall| - Force apps to use |tor_project| +* |f-droid_orweb| - Privacy-enhanced browser +* |f-droid_password_store| - Manage your passwords +* |f-droid_plumble| - Voice chat for |mumble| servers +* |f-droid_practice_hub| - Tools for musicians +* |f-droid_port_authority| - Port scanner +* |f-droid_quickdic| - Offline translation dictionary +* |f-droid_sms_backup| - Backup |abbr_sms| and call logs to |abbr_imap| +* |f-droid_silence| - Send encrypted text messages (SMS/MMS) +* |f-droid_satstat| - Signal Generator for tablets +* |f-droid_signal_generator| - Signal Generator for tablets +* |f-droid_snoopsnitch| - Check mobile network security +* |f-droid_syncthing| - File synchronization +* |f-droid_termux| - Terminal emulator with packages +* |f-droid_transportr| - Public Transport Companion +* |f-droid_tryton| - Enterprise resource management +* |f-droid_twidere| - Microblogging client +* |f-droid_vlc| - Media player +* |f-droid_weechat| - Internet relay chat +* |f-droid_world_weather| - View weather forecast + +Guardian Project +________________ +| The |guardian_project| offers a |guardian_project_repository| for |f-droid| with free/ libre Android applications that are being developed by a team of volunteers and are all evolving around the matter of strong encryption for communication. +| One of the more notable projects was to bring |tor_project| to Android. +| The |guardian_project| has more buns in the oven though. Some are directly available through |f-droid|, others only through their own |guardian_project_repository|. This is a collection of the currently available projects: + +* |guardian_project_chatsecure| - A free and open source messaging app that features |otr| encryption over |abbr_xmpp| +* |guardian_project_orbot| - A free |abbr_proxy| app that empowers other apps to use the internet more securely +* |guardian_project_camerav| - The easiest way to capture and share verifiable photos and video proof on a smartphone or tablet, all the while keeping it entirely secure and private +* |guardian_project_obscuracam| - A photo and video app for Android that keeps certain information private. +* |guardian_project_pixelknot| - An Android application that allows users to hide short text-based messages in photographs and share them across trusted channels. + +Conclusion +__________ +Yay! + +* I don't need to buy a new phone each year, because my battery got eaten by malicious apps that I can't disable. +* I can use an Android device without Google +* I can use an Android device without vendor bloat +* I can use an Android device as root and modify it the way I want it +* I can choose to only install free apps + +.. |otr| raw:: html + + OTR + +.. |kodi| raw:: html + + Kodi + +.. |mpd| raw:: html + + MPD + +.. |openvpn| raw:: html + + OpenVPN + +.. |mumble| raw:: html + + mumble + +.. |free_software| raw:: html + + free/libre software + +.. |tor_project| raw:: html + + Tor + +.. |guardian_project_pixelknot| raw:: html + + Pixelknot + +.. |guardian_project_obscuracam| raw:: html + + ObscuraCam + +.. |guardian_project_camerav| raw:: html + + CameraV + +.. |guardian_project_orbot| raw:: html + + Orbot + +.. |guardian_project_chatsecure| raw:: html + + ChatSecure + +.. |f-droid_conversations| raw:: html + + Conversations + +.. |f-droid_droidshows| raw:: html + + DroidShows + +.. |f-droid_kontalk| raw:: html + + Kontalk + +.. |f-droid_logical_defence| raw:: html + + Logical Defence + +.. |f-droid_mpdroid| raw:: html + + MPDroid + +.. |f-droid_mupdf| raw:: html + + MuPDF + +.. |f-droid_open_camera| raw:: html + + Open Camera + +.. |f-droid_port_authority| raw:: html + + Port Authority + +.. |f-droid_practice_hub| raw:: html + + Practice Hub + +.. |f-droid_quickdic| raw:: html + + QuickDic + +.. |f-droid_satstat| raw:: html + + SatStat + +.. |f-droid_signal_generator| raw:: html + + Signal Generator + +.. |f-droid_silence| raw:: html + + Silence + +.. |f-droid_world_weather| raw:: html + + World Weather + +.. |f-droid_snoopsnitch| raw:: html + + SnoopSnitch + +.. |f-droid_weechat| raw:: html + + Weechat + +.. |f-droid_vlc| raw:: html + + VLC + +.. |f-droid_transportr| raw:: html + + Transportr + +.. |f-droid_syncthing| raw:: html + + Syncthing + +.. |f-droid_twidere| raw:: html + + Twidere + +.. |f-droid_tryton| raw:: html + + Tryton + +.. |f-droid_sms_backup| raw:: html + + SMS Backup+ + +.. |f-droid_password_store| raw:: html + + Password Store + +.. |f-droid_orweb| raw:: html + + Orweb + +.. |f-droid_orwall| raw:: html + + orWall + +.. |f-droid_orbot| raw:: html + + Orbot + +.. |f-droid_osmand| raw:: html + + OsmAnd~ + +.. |f-droid_openvpn_for_android| raw:: html + + OpenVPN for Android + +.. |f-droid_openvpn_settings| raw:: html + + OpenVPN Settings + +.. |f-droid_oandbackup| raw:: html + + oandbackup + +.. |f-droid_owncloud_notes| raw:: html + + ownCloud Notes + +.. |f-droid_owncloud_news| raw:: html + + ownCloud News + +.. |f-droid_owncloud| raw:: html + + ownCloud + +.. |f-droid_openkeychain| raw:: html + + OpenKeychain + +.. |f-droid_k-9_mail| raw:: html + + K-9 Mail + +.. |guardian_project_repository| raw:: html + + repository + +.. |guardian_project| raw:: html + + Guardian Project + +.. |f-droid_irssi_connectbot| raw:: html + + Irssi ConnectBot + +.. |f-droid_connectbot| raw:: html + + ConnectBot + +.. |f-droid_firefox| raw:: html + + Firefox + +.. |f-droid_kore| raw:: html + + Kore + +.. |f-droid_f-droid| raw:: html + + F-Droid + +.. |f-droid_plumble| raw:: html + + Plumble + +.. |f-droid_termux| raw:: html + + Termux + +.. |ardour| raw:: html + + Ardour + +.. |f-droid_opentasks| raw:: html + + OpenTasks + +.. |f-droid_davdroid| raw:: html + + DAVdroid + +.. |f-droid_cadroid| raw:: html + + CAdroid + +.. |f-droid_ardroid| raw:: html + + Ardroid + +.. |f-droid_antennapod| raw:: html + + AntennaPod + +.. |f-droid_afwall| raw:: html + + AFWall+ + + +.. |cyanogenmod_ville_install| raw:: html + + install section + +.. |cyanogenmod_ville_development_channel| raw:: html + + development channel + +.. |cyanogenmod_ville_release_channel| raw:: html + + release channel + +.. |htc_one_s_last_update| raw:: html + + last update + +.. |htc_one_s| raw:: html + + HTC One S + +.. |cyanogenmod_ville| raw:: html + + ville + +.. |f-droid| raw:: html + + F-Droid + +.. |google_play| raw:: html + + Google Play + +.. |28c3_doctorow| raw:: html + + Cory Doctorow at 28C3 + +.. |cyanogenmod| raw:: html + + CyanogenMod + +.. |cyanogenmod_supported_devices| raw:: html + + supported devices + +.. |abbr_nfc| raw:: html + + NFC + +.. |abbr_ssh| raw:: html + + SSH + +.. |abbr_xmpp| raw:: html + + XMPP + +.. |abbr_vpn| raw:: html + + VPN + +.. |abbr_sms| raw:: html + + SMS + +.. |abbr_imap| raw:: html + + IMAP + +.. |abbr_proxy| raw:: html + + proxy + diff --git a/posts/2016/201604-linux-audio-conference-2016.rst b/posts/2016/201604-linux-audio-conference-2016.rst new file mode 100644 index 0000000..9f78ee9 --- /dev/null +++ b/posts/2016/201604-linux-audio-conference-2016.rst @@ -0,0 +1,376 @@ +.. title: Linux Audio Conference 2016 +.. date: 2016-04-15 23:00 UTC+02:00 +.. modified: 2015-04-16 23:00 UTC+02:00 +.. tags: ccc, voc, c-base, berlin, linuxaudio, linux audio conference, minilac, pro-audio, electronic studio, streaming +.. category: events +.. slug: linux-audio-conference-2016 +.. summary: On how I ended up organizing a not-for-profit, small-scale conference for pro-audio developers for Linux with close to zero budget +.. authors: David Runge + +The conference +______________ +| The |linux_audio_conference| is actually a quite old concept by now. Started as a small Linux Audio user group meeting at LinuxTag back in 2002, the conference more and more developed into a multi-national event, thanks to people such as Frank Neumann (who by the way initially had a *"hacker meeting"* in mind) and places like the |zkm|. +| As more universities hosted it, its academic side strengthened, leading to proper |abbr_proceedings|, paper and poster presentations. +| Generally speaking it has also always been a place to present software, do workshops to show people how to use software and try it out - suited for developers, users and interested alike! +| Another nice aspect that evolved over the years is the concept of the *"Linux Sound Night"*, giving the stage to the artists to present their pieces or perform live. +| There's obviously a lot more to the history of the |linux_audio_conference| (which is no wonder after such a long time!), than I could elaborate on. +| By now the |lac| has taken place in many different countries: Germany, Australia, Italy, The Netherlands, Ireland, USA and Austria. +| +| This is where I offer you the following three choices: +| + +* |abbr_tldr|, *"I don't care about the pain, take me to the good times!"*: Just skip the *"how"* and go to `the results `_ +* |abbr_tldr|, *"I only care about the pain"*: Watch |video_lac_is_dead_long_live_minilac| +* *"I'm all up for pain (and good times, for that matter)"*: Read on! Additionally watch |video_lac_is_dead_long_live_minilac| + + +Last year +_________ +| Last year `I went to LAC in Mainz `_, that |albert_graef| so beautifully orchestrated, to talk about my pro-audio setup for |archlinux| and install `The Sound Of People `_. +| I really liked the conference, the getting together, the artists and developers involved and the overall free access approach. I had nice chats with many of the visitors and vendors showing their hard- and software (such as Harry van Haaren (|openav|), Gianfranco Ceccolini (|mod_devices|), Heiko Weinen (then |bitwig|) amongst many others). +| As I knew, that the |tu-berlin| had hosted the |lac| back in 2007, initiated by people around |marije_baalman|, I spoke to |albert_graef| about the processes involved around doing the conference. I liked the idea of bringing it back to Berlin after such a long time. +| +| So after |lac| several things began to emerge: |linuxaudio.berlin| and my desire to get |tu-berlin| aboard to do the conference in 2016. +| Initially, carried by a post-|lac| drive the Linux Audio Users Berlin group was quite large and we setup monthly meetings for it at |c-base|. +| The idea of doing the conference was put forward and I finally got around introducing it to my new work place, the |electronic_studio|, part of the |audio_communication_group|. Responses weren't negative (which is university-speak for *"great idea, but we'll only do it, if it doesn't mean work for us"*), which wasn't a big deal at that time, as a subset of interested people at |linuxaudio.berlin| seemed willing to get truly involved and mainly organize the event. +| It was suggested to me to get the |udk-berlin| in for taking care of concert venues and strengthening the team. Their initial response was very positive. +| So everything looked, as if it could work out nicely and I announced to the former |lac| organizers that this boat would float (at that time also |JMU| (St. Etienne, France) and |CCRMA| (Stanford, USA) were racing to get their facts straight). +| Horrible mistake. +| +| After a few weeks of limbo it became quite clear, that none of the universities (in Berlin) asked for involvement were actually ready to commit to the task. While |tu-berlin| opted out, |udk-berlin| was never heard of again. +| Discussing this with the folks of |linuxaudio.berlin| we aimed at trying the approach of a sponsor-based event, using |c-base| and the rooms of |eti| (which obviously required rent). +| Not being able to rely upon institutional expertise in doing this sort of event, we tried to stem the workload with the few people we had available. Process was slow (as you can imagine). +| We tried working out all needed facts (using the |github| issue tracking system for it: |minilac_issues|) and promoting the undertaking (at nice places like `CCCamp2015 `_) with the ressources available. +| Quite early I asked the |voc| about a possible involvement and although their schedule is filling up crazy fast, they liked the idea of a small not-for-profit conference about pro-audio, open-source software and/ on Linux in Berlin (as most of them live here). +| As the year was already progressed and we were running out of time, it of course got harder to find sponsors (and **it is always hard to find sponsors for events**, especially if those are about free software and Linux and *all that other hippie stuff*). +| We also applied for cultural funding by the |hauptstadtkulturfonds|, a process that needs a painful load of details and paperwork and an applicational process for which most companies hire other companies to do it. To anyone who was never involved with such a fonds, let me tell you: It takes a loooooooooong time until you hear back from them. +| At the end of the year we still hadn't found any sponsors for the event. Hard times. +| + +This year +_________ +| Last year ended kind of depressing (in terms of the |lac|) and this year would start in the same vein. Our last straw the |hauptstadtkulturfonds| turned us down and it became clear, that the conference would not happen. We had failed and didn't feel too great about it. +| I communicated our situation to the former organizers (who always lent an ear, if they had the time and tried to give advice). +| For the usual process of paper reviews, etc. it was already too late anyways (and we rightfully received some heat from the community for that), but we had the feeling we couldn't just give up and abandon ship yet. +| |linuxaudio.berlin| suffered quite a bit under these undertakings and I'm sure that many were just too annoyed about the topic to ever show up again. +| Sorry for that... +| +| Heiko '|riot|' Weinen had the great idea of just shrinking it down to a size we can handle ourselves: |c-base| +| Another member from |linuxaudio.berlin|, Daniel '|excds|' Swärd, got involved, after others, that had helped along the way either got to busy with their own things or just couldn't be bothered anymore, helping define its form further. +| Eventually this meant turning the ship around, cutting down on all the things we couldn't deal with (paper presentations, big concerts) and announcing, that |mail_lac_is_dead_long_live_minilac|. +| We populated a |mediawiki| instance with useful information about the location and what we had in mind, using templates (leading to *template-ception*), giving the community the possibility to sign up and generate their own content, setting up their workshops, lectures and hacking sessions themselves. Overall a nice process, in my opinion, lifting much of the work from our shoulders (as there was no review process involved to begin with). +| I got the |voc| back aboard and |edgar_berdahl| happened to become our keynote lecturer. Suddenly we had an ace up our sleeve (at least at that time it felt like it... well, I guess it still feels like that). +| +| After being rather nicely informed by |nils_gey| about the misinformation taking place on |linuxaudio.berlin| on the date and location of |minilac|, I took it upon me to change the website from a |wordpress| driven mess, that was suffering from the now and again dying MySQL server on that machine, to a |pelican| based website. Just one of the many last-minute things to do. +| +| Two weeks before the start of |minilac| the wiki was attacked by a wave of spammers and I had to deal with the unpleasant work of deleting the pages of 500 spambots, blocking their accounts and strengthening the (hastily setup) security measures for the wiki. Lesson learnt. +| Apart from the spambots the wiki also entailed another issue, that has to do with the way the |voc| operates. Being free-time professionals, they usually have conferences use |frab| for dealing with the content, which is an all-in-one conference management system, that generates all things needed for streaming, |info-beamer|, interfacing the |engelsystem| and generating a |fahrplan| (especially the latter is super helpful during large events to keep track of what is happening where, as there's also an Android App available for it). +| This meant parsing the information from our wiki and generating the needed |abbr_xml| and |abbr_json| files ourselves. Pretty painful, but we got it done (well, apart from triggering a bunch of bugs in the |voc|'s |voc_schedule_validator|, which lead to never being able to generate a proper |fahrplan|). +| Last minute fixes for the |info-beamer| were needed as well, but thanks to the |voc|, we got them done just in time! +| Meanwhile I prepared some hardware to be lent from the |electronic_studio| and the insurance for the |voc| equipment (to be used in combination with already available |c-base| hardware). +| +| Just a week before |minilac|, I organized the `MSSW Überworkshop `_, so everything felt pretty squeezed on the Friday before the event. I fetched the |electronic_studio| hardware and the |voc| equipment from |cccb| and off we went for a crazy evening of setting up hardware and doing last minute fixes, while some attendees already showed up for the meet-and-greet and watching a |spacex| |spacex_space_shuttle_launch|. +| + +miniLAC +_______ + +Saturday +-------- +|minilac_link| + +| The first |minilac| day started a bit rough (after only a few hours of sleep), as we had some minor setup issues, that got solved just in time. +| I guess that's one of the reasons for our |video_opening| being a little off. From then on it just kept getting better, though! +| +| |edgar_berdahl| (of Louisiana State University) gave an excellent keynote lecture about his approaches in physical modelling with Faust (|video_open_source_haptics_for_music|), followed by last year's organizer |albert_graef|, who talked about |video_plugin_programming_with_faust|. +| It would be useless redundancy to reiterate the |minilac_schedule| at this point. Find the things you like and watch them: |voc_minilac_videos| +| +| The Saturday also offered our small version of the |linux_sound_night|, starting with an amazing outside performance by |asphyxia_collective| and ending in an open jam session. +| At this point I have to apologize to |fredrik_olofsson| again, whose set we so awefully bombed that night by being too tired and not taking care of the stage and its slots the way we should have. I hope you're not still mad at us! +| Luckily, that day I was not the last person to leave |c-base|. I'm quite sure some stayed very long. +| + +Sunday +------ +| Just as on Saturday we had a weird start. A power outage around 09:30 took down the streaming equipment and we had to set things up until our own talk around 10:00. +| Despite this minor annoyance, we had a good time talking about the future of |lac| in |video_lac_is_dead_long_live_minilac|, giving hints to next year's location and improvement suggestions. +| If you'd like to work with us on the relaunch of the websites and planning tools of |linux_audio|, come join our newly founded association on |github|: +| |github_linux_audio| +| +| The day went on with nice lectures, workshops and hacking sessions such as |video_public_domain_project|, |video_getting_to_know_yoshimi| and |video_bela|. +| Again, just pick things from the |minilac_schedule| and find the videos you like: |voc_minilac_videos| +| +| After us saying goodbye (|video_closing|), we started dismantling the whole thing and I'm glad a bunch of people from |c-base| and |linuxaudio.berlin| came to help. I was pretty much destroyed at that point! +| + +Photos +------ +.. figure:: /images/2016/minilac/img_5161.jpg + :alt: Group photo (stage 3 sillification) + + Group photo (stage 3 sillification) + +| Some pictures from the event made it to my new photos page: `photos from miniLAC2016 `_ +| + +Lessons learnt +______________ +These are the lessons learnt from doing this event: + +* Start planning **as early, as humanly possible** (it will take forever anyways) +* Make sure your affiliation actually wants to do this (**ink in blood!**) +* Put a lot of effort into the planning phase (you'll forget something nonetheless, but it'll give a hint of security) +* Read the FAQ +* Make a list of things you can and can not provide +* Do not rely on external funding (sponsoring, cultural funding) +* Outsource information! |mediawiki| is a good choice! +* Secure (protect from spam) all your resources properly! +* Ask the |voc| to do your streaming! They're awesome and they're pros! +* Use |frab| to manage your conference! +* According to |murphys_law|, shit **will** inevitably hit the fan, (e.g. if your fridge is prone to break, it **will** break a day before the conference) - plan for the worst! + +| All in all I'm surprised what got accomplished with a budget of only 400€ (all later covered by donations from attendees and |c-base|). +| Of course this meant **a lot of work** and would not have been possible without many volunteers! Hackerspaces and the communities around them seem very well suited for this type of event though. +| Also, the general layout of all former conference editions made sure, that attendees are not from a scientific background exclusively, which I think would move the event into a direction that is undesirable. +| I had a great time and I really like the |linux_audio| community in all of its facettes. I'm looking forward to next year (possibly in St. Etienne), but luckily I won't be organizing then! ;-) + +.. |riot| raw:: html + + riot + +.. |excds| raw:: html + + excds + +.. |spacex_space_shuttle_launch| raw:: html + + space shuttle launch + +.. |spacex| raw:: html + + SpaceX + +.. |engelsystem| raw:: html + + Engelsystem + +.. |wordpress| raw:: html + + Wordpress + +.. |pelican| raw:: html + + Pelican + +.. |nils_gey| raw:: html + + Nils Gey + +.. |minilac_schedule| raw:: html + + schedule + +.. |video_closing| raw:: html + + Closing + +.. |video_bela| raw:: html + + BELA + +.. |video_getting_to_know_yoshimi| raw:: html + + Yoshimi + +.. |video_public_domain_project| raw:: html + + The Public Domain Project + +.. |github_linux_audio| raw:: html + + https://github.com/linuxaudio + +.. |linux_audio| raw:: html + + Linux Audio + +.. |minilac_link| raw:: html + + + +.. |fredrik_olofsson| raw:: html + + Fredrik Olofsson + +.. |asphyxia_collective| raw:: html + + Asphyxia Collective + +.. |linux_sound_night| raw:: html + + Linux Sound Night + +.. |voc_minilac_videos| raw:: html + + https://media.ccc.de/c/minilac16 + +.. |murphys_law| raw:: html + + Murphy's Law + +.. |video_plugin_programming_with_faust| raw:: html + + Plugin programming with Faust + +.. |video_open_source_haptics_for_music| raw:: html + + Open-Source Haptics for Music + +.. |edgar_berdahl| raw:: html + + Edgar Berdahl + +.. |video_opening| raw:: html + + Opening + +.. |cccb| raw:: html + + CCCB + +.. |voc_schedule_validator| raw:: html + + schedule validator + +.. |fahrplan| raw:: html + + Fahrplan + +.. |info-beamer| raw:: html + + info-beamer + +.. |frab| raw:: html + + frab + +.. |minilac| raw:: html + + miniLAC + +.. |mediawiki| raw:: html + + Mediawiki + +.. |github| raw:: html + + Github + +.. |minilac_issues| raw:: html + + LAC16 issues + +.. |mail_lac_is_dead_long_live_minilac| raw:: html + + "LAC is dead! Long live miniLAC!" + +.. |video_lac_is_dead_long_live_minilac| raw:: html + + LAC is dead! Long live miniLAC! + +.. |hauptstadtkulturfonds| raw:: html + + Hauptstadtkulturfonds + +.. |voc| raw:: html + + VOC + +.. |eti| raw:: html + + ETI + +.. |CCRMA| raw:: html + + CCRMA + +.. |JMU| raw:: html + + Jean Monnet Université + +.. |udk-berlin| raw:: html + + UdK Berlin + +.. |audio_communication_group| raw:: html + + audio communication group + +.. |electronic_studio| raw:: html + + Electronic Studio at TU-Berlin + +.. |c-base| raw:: html + + c-base + +.. |linuxaudio.berlin| raw:: html + + linuxaudio.berlin + +.. |marije_baalman| raw:: html + + Marije Baalman + +.. |tu-berlin| raw:: html + + TU Berlin + +.. |bitwig| raw:: html + + Bitwig + +.. |mod_devices| raw:: html + + MOD Devices + +.. |openav| raw:: html + + OpenAV + +.. |archlinux| raw:: html + + Arch Linux + +.. |albert_graef| raw:: html + + Albert Gräf + +.. |lac| raw:: html + + LAC + +.. |linux_audio_conference| raw:: html + + Linux Audio Conference + +.. |zkm| raw:: html + + ZKM + +.. |abbr_proceedings| raw:: html + + proceedings + +.. |abbr_tldr| raw:: html + + tl;dr + +.. |abbr_xml| raw:: html + + XML + +.. |abbr_json| raw:: html + + JSON + diff --git a/posts/2016/201604-mssw.rst b/posts/2016/201604-mssw.rst new file mode 100644 index 0000000..1ad578f --- /dev/null +++ b/posts/2016/201604-mssw.rst @@ -0,0 +1,240 @@ +.. title: Modular Synth Selbstbau Workshop +.. date: 2016-04-06 22:00 UTC+02:00 +.. modified: 2016-04-06 22:00 UTC+02:00 +.. tags: c-base, berlin, eurorack, modular, synthesizer, befaco, 000, rebel technology, bastl instruments, 4ms, lithium, battery, apple +.. category: events +.. slug: modular-synth-selbstbau-workshop +.. summary: How I got affiliated with a bunch of modular synthesizer vendors to do awesome workshops at a space station. +.. authors: David Runge + +In the beginning there was one +______________________________ +| Last year, when I started building |abbr_eurorack|, what eventually took it's peak of craziness in `building my own suitcase `_, I got to know the great folks at |befaco|, a Spanish group of friends that started designing and eventually selling their own modules. +| Back then, the |nk| still existed as a venue and it was host to many concerts alongside the DIY workshops of |befaco| (taken care of by |florian_hanisch|). +| When it |nk-closing-statement|, I had already spent some time at |c-base| and found it to be a great place for synthesizer workshops (I mean, what better place for synths is there, but a space station? Well, maybe cats *and* synthesizers at a space station!). +| + +But wait, there's more! +_______________________ +| Just how it is with the meeting of like-minded people: They tend to find more of a kind! +| |befaco| are long time friends with |victor_mazon|, who just started to design his own modules under the moniker |zero_zero_zero| and spent workshops together with |rebel_tech| at |london_hackspace|. +| All of these people are most excellent, provide open hardware and I am very happy to now have them over at |c-base| for doing their workshops! +| So by the end of 2015, |zero_zero_zero|, |befaco| and |rebel_tech| started as a joint effort under the name |mssw|. +| + +The Überworkshop +________________ +.. figure:: /images/mssw_workshop.jpg + :alt: The seminar room with an ongoing modular workshop. + + The |c-base| seminar room with workshop attendees, vendors |befaco|, |four_ms|, |bastl_instruments| and the usual suspects like Jeremy Bernstein + +| Last weekend for the first time the three were in Berlin at |c-base| at the same time (due to |superbooth|), accompanied by the great folks of |bastl_instruments| and |four_ms|. +| Although everyone was pretty much destroyed from three days of listening to noise and talking to synthesizer people, they still managed to squeeze out a workshop day with all vendors combined! Hence, the **"Überworkshop"**. +| I somehow finished the |bastl_instruments_grandpa| and |bastl_instruments_little_nerd| of |bastl_instruments| (really looking forward to spend some time with those the coming weeks!) and a |befaco_power_bus| for the |c-base_soundlab| (gotta start somewhere, right?). +| + +The suicidal machine +____________________ +| Then this happened: +| +| |mssw_burning_laptop| +| +| Some short notes on the burning laptop: + +* Laptops use |lithium| based batteries. If they get too hot, they can burn! +* |lithium_reacts_with_water| (so don't ever spill water on Lithium or a laptop for that matter) +* Extinguishing the burning laptop with water worked here, because most of the |lithium| had reacted already and the water took away the oxigen for the reaction (the burning). This could have gone another way! +* Burning laptops generate **A LOT** of smoke. Leave the room! (Your furniture is most likely ruined by now) + +.. figure:: /images/mssw_workshop_macbook_suicide.jpg + :alt: A burnt up MacBook after its Lithium battery got too hot. + + On Sunday, 5th April 2016 at approximately 19:27 this MacBook gave its life for a future without Apple products. + +Poor Martin of |bastl_instruments| lost his laptop (and I heard his iPhone died just a week later). + +* Don't use Apple devices! +* Backup regularly! + +The showcases +_____________ + +| Not even the |ijon_twitter_macbook_suicide| could keep us from having a nice showcase evening together with Dan Snazelle of |snazzyfx| and the |mssw| crew afterwards! +| As special (self-invited) guest, we had |der_warst| aka. |simon_schaefer| (another very interesting acquaintance from 2014's |bended_realities|) with his awesome DIY circuit bending hardware. +| + +.. figure:: /images/mssw_workshop_befaco_showcase.jpg + :alt: |befaco| showcasing their equipment. + + Diego and Manu of |befaco| showcasing their equipment. + +| |befaco| did a nice set, of which I unfortunately only recorded beginning and end (the missing chunk is barely noticeably though ;-) ) due to running out of space on my |tascam_dr40|. Next time I'll be better prepared! |showcase_befaco| +| +| In return I don't have a picture of Martin Klang (|rebel_tech|), but hey... a nice recording will do: |showcase_rebeltech| +| +| With |zero_zero_zero| it's unfortunately the same, although they played the longest set: |showcase_000| + +.. figure:: /images/mssw_workshop_snazzy_showcase.jpg + :alt: |snazzyfx| showcase + + Dan Snazelle of |snazzyfx| showcasing his equipment. + +| Dan, much more than the others, went into a danceable direction. Listen for yourself: |showcase_snazzyfx| + +.. figure:: /images/mssw_workshop_jan_wilhelm_showcase.jpg + :alt: Jan Wilhelm showcase + + Jan Wilhelm (the |superbooth| party machine) showcasing his equipment. + +| Jan Wilhelm seems to have been all partied out from |superbooth| (where he apparently did techno sets all day long), as his showcase was rather experimental: |showcase_janwilhelm| +| As mentioned before, |simon_schaefer| (aka. |der_warst|) popped by, when everything was nearly over and introduced his magnificent multi-use telephone: |showcase_derwarst| + +| All in all the event prooved to be great fun, but also meant a lot of work and little sleep. Not something one organizes on a weekly basis (I hope). + +.. |befaco| raw:: html + + Befaco + +.. |befaco_power_bus| raw:: html + + power bus + +.. |c-base| raw:: html + + c-base + +.. |c-base_soundlab| raw:: html + + soundlab + +.. |florian_hanisch| raw:: html + + Florian Hanisch + +.. |ijon_twitter_macbook_suicide| raw:: html + + suicide of a MacBook + +.. |lithium| raw:: html + + Lithium + +.. |lithium_reacts_with_water| raw:: html + + Lithium reacts with water + +.. |mssw_burning_laptop| raw:: html + + + +.. |nk| raw:: html + + NK + +.. |nk-closing-statement| raw:: html + + closed down + +.. |victor_mazon| raw:: html + + Victor Mazón Gardoqui + +.. |zero_zero_zero| raw:: html + + 000 + +.. |rebel_tech| raw:: html + + Rebel Technology + +.. |london_hackspace| raw:: html + + London Hackspace + +.. |mssw| raw:: html + + Modular Synth Selbstbau Workshop + +.. |superbooth| raw:: html + + Superbooth + +.. |bastl_instruments| raw:: html + + Bastl Instruments + +.. |bastl_instruments_grandpa| raw:: html + + grandPA + +.. |bastl_instruments_little_nerd| raw:: html + + LITTLE NERD + +.. |four_ms| raw:: html + + 4MS + +.. |snazzyfx| raw:: html + + SnazzyFX + +.. |der_warst| raw:: html + + der Warst + +.. |simon_schaefer| raw:: html + + Simon Schäfer + +.. |tascam_dr40| raw:: html + + Tascam DR-40 + +.. |bended_realities| raw:: html + + Bended Realities + +.. |showcase_befaco| raw:: html + + + +.. |showcase_rebeltech| raw:: html + + + +.. |showcase_000| raw:: html + + + +.. |showcase_snazzyfx| raw:: html + + + +.. |showcase_janwilhelm| raw:: html + + + +.. |showcase_derwarst| raw:: html + + + +.. |abbr_eurorack| raw:: html + + Eurorack + diff --git a/posts/2016/201609-darmstadt.rst b/posts/2016/201609-darmstadt.rst new file mode 100644 index 0000000..63b6132 --- /dev/null +++ b/posts/2016/201609-darmstadt.rst @@ -0,0 +1,430 @@ +.. title: Darmstadt 2016 +.. date: 2016-09-27 22:00 UTC+02:00 +.. modified: 2016-09-28 22:00 UTC+02:00 +.. tags: darmstadt, supercollider, bowelyzer, neue musik, annesley black, tolerance stacks, international summer courses for new music +.. category: events +.. slug: darmstadt-2016 +.. summary: I created a SuperCollider based audio analysis tool for a piece by |website-annesley_black|, that was premiered at the |48th_international_summer_courses_for_new_music| in Darmstadt. +.. authors: David Runge + +Summer courses +______________ + +| In its 48th edition, the |international_summer_courses_for_new_music| were taking place in Darmstadt this year with a wide |international_summer_courses_for_new_music-program|. +| Initiated just after World War II by Wolfgang Steinecke (then head of Department of Arts and Culture, in a city that was pretty much completely destroyed during the war), the |wiki-darmstadt_school| has risen to glory not only because of its famous seminarists such as Karlheinz Stockhausen, but mainly because of its diversity, international flavor and rehabilitation of art that was tainted by the Third Reich. +| I am happy to have been a part of this year's spectacle, although I didn't really see much aside from the rehearsals and premiere of the piece I came to help establish: |tolerance_stacks| by |website-annesley_black|. +| + +Tolerance Stacks +________________ +| Annesley's piece is a very funny and diverse multi-channel multimedia experience, that got its name from the process of decaying electronic parts, that change their behavior over time, in turn changing the workflow of the system they are used in (by |wiki-tolerance_analysis| of the parts). +| She was confronted with this undesired behavior while trying to get old music gear of |wiki-hermann_heiss| fixed during research on the late |wiki-twelve-tone_technique| and |wiki-electronic_music| composer. Initially, some of his synthesizers were meant to be used in the piece, but in the end only two |wiki-teac| tape machines made their way into it, as the rest was unrepairable. +| |tolerance_stacks| (try the |tolerance_stacks-german| for a better description and biographies) gives a funky perspective on these *relics from the olden days* by commenting with historical markerpoints, rap, unconventional tools (|wiki-etch_a_sketch|) and instruments (ideogrammophon, no-input-mixer, turntables), constant positional changes of the musicians, alongside a very neat video projection by |website-lutz_garmsen|, showcasing the bowels of old machines and tools. +| + +Collidin' +_________ +| While the musicians involved did a very fine job rehearsing the piece in such a short amount of time, a pressure project like this obviously leads to friction, which luckily is released after the premiere has taken place. +| I for my part had a relatively relaxed job behind the mixing desk, as I monitored a piece of software I wrote to feed Lutz's |website-vvvv| with |wiki-osc| messages derived from the various audio inputs coming from musicians and microphones. +| As a little inside joke I baptized this |website-supercollider| application |bowelyzer|. It is able to monitor up to *n* channels (also *n* times each) and send |wiki-osc| messages depending on the settings you choose. It uses |abbr_json| for saving and loading its settings. +| These are the settings we used for |tolerance_stacks|: + + .. code:: javascript + + { + "forwardAddress": "192.168.2.2", + "forwardPort": 9998, + "hubAddress": "127.0.0.1", + "hubPort": 57120, + "synthServerAddress": "127.0.0.1", + "synthServerPort": 57110, + "controls": { + "Comp": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.1, + "freqRange": [ 100, 14000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.01, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 14000, + "pitchMedian": 1, + "pitchMinFreq": 80, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Drms": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.093, + "freqRange": [ 60, 16000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.16, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 14000, + "pitchMedian": 1, + "pitchMinFreq": 60, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Ideo": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.1, + "freqRange": [ 60, 14000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.01, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 14000, + "pitchMedian": 1, + "pitchMinFreq": 60, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Moog": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.1, + "freqRange": [ 30, 16000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.01, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 16000, + "pitchMedian": 1, + "pitchMinFreq": 30, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20.1 + }, + "NIM": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.1, + "freqRange": [ 60, 13000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.01, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 13000, + "pitchMedian": 1, + "pitchMinFreq": 60, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Pub": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.01, + "freqRange": [ 150, 16000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.04, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 12000, + "pitchMedian": 1, + "pitchMinFreq": 60, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Revox": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.1, + "freqRange": [ 63.25, 12000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.01, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 14000, + "pitchMedian": 1, + "pitchMinFreq": 60, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Sax1": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.184, + "freqRange": [ 100, 10000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.11, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 10000, + "pitchMedian": 1, + "pitchMinFreq": 100, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Sax2": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.139, + "freqRange": [ 60, 14000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.17, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 11000, + "pitchMedian": 1, + "pitchMinFreq": 200, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Sop": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.184, + "freqRange": [ 100, 12000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 0.3, + "hfWaitTime": 0.04, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.05, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 14000, + "pitchMedian": 1, + "pitchMinFreq": 100, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + }, + "Ttb": { + "active": true, + "amplitudeAttackTime": 0.01, + "amplitudeReleaseTime": 0.01, + "freqRange": [ 50, 16000 ], + "hfFoote": 0, + "hfHainsworth": 1, + "hfThreshold": 2.706, + "hfWaitTime": 0.349, + "onlyForwardOnNewPitch": false, + "pitchAmpThreshold": 0.03, + "pitchDownSample": 1, + "pitchExecFreq": 100, + "pitchInitFreq": 440, + "pitchMaxBinsPerOctave": 16, + "pitchMaxFreq": 16000, + "pitchMedian": 1, + "pitchMinFreq": 50, + "pitchPeakThreshold": 0.5, + "sendReplyFreq": 20 + } + }, + "inputs": { + "Comp": 1, + "Drms": 15, + "Ideo": 11, + "Moog": 17, + "NIM": 13, + "Pub": 2, + "Revox": 0, + "Sax1": 10, + "Sax2": 12, + "Sop": 14, + "Ttb": 16 + } + } + +| If you are interested, go and check out its |bowelyzer-about| page. +| +| So, while Lutz was sitting in front of the stage, dealing with a bunch of camera-enhanced |website-rpi3|, that in turns filmed parts of the musicians' work, |bowelyzer| sent its messages, which were used to morph and move the video pieces, over |abbr_lan|. +| In the meantime we were able to communicate through a local |website-etherpad| instance. +| Overall a very nice process, that got more visually interesting by the minute (although I must admit I liked the second performance's visuals the best). +| + +Aftermath +_________ +| The premiere has been recorded (video and audio) and there is a video in the making. As soon as I know more, I will post an update to this. +| + +Sidestory +_________ +| Me having less to do during the rehearsal process didn't mean less stress for me (apart from the programming marathon and the calibration process on-site). +| As soon as I got on the train in Berlin, I realized, that I had forgotten my power adaptor for my |wiki-lenovo_thinkpad_x220|. +| Luckily, this is a very common laptop and has an even more common power adaptor. +| So, instead of going crazy, I spent half an hour on the train to communicate with the nice folks at |website-chaos_darmstadt|, whose hackspace |website-trollhoehle| (|twitter-trollhoehle|) I wanted to visit anyways. +| Turns out, they have a heap of the power adaptors just lying around and their new spot was 100m away from |website-centralstation|, where |tolerance_stacks| was rehearsed and performed. +| This way I ended up with a borrowed power adaptor for my laptop, some nice acquaintences from one of the many |website-ccc| bubbles and a great spot to decompress. Thanks again, folks! +| If you're visiting Darmstadt, make sure to go by their space in the late afternoon/ evening. It's worth a peak. +| + +.. |website-annesley_black| raw:: html + + Annesley Black + +.. |48th_international_summer_courses_for_new_music| raw:: html + + 48th International Summer Courses for New Music + +.. |international_summer_courses_for_new_music| raw:: html + + International Summer Courses for New Music + +.. |international_summer_courses_for_new_music-program| raw:: html + + program + +.. |tolerance_stacks| raw:: html + + Tolerance Stacks + +.. |wiki-darmstadt_school| raw:: html + + Darmstadt School + +.. |wiki-karlheinz_stockhausen| raw:: html + + Karlheinz Stockhausen + +.. |wiki-tolerance_analysis| raw:: html + + accumulating tolerances + +.. |wiki-hermann_heiss| raw:: html + + Hermann Heiß + +.. |wiki-twelve-tone_technique| raw:: html + + Twelve-tone technique + +.. |wiki-electronic_music| raw:: html + + Electronic music + +.. |wiki-teac| raw:: html + + TEAC + +.. |wiki-etch_a_sketch| raw:: html + + Etch A Sketch + +.. |website-lutz_garmsen| raw:: html + + Lutz Garmsen + +.. |tolerance_stacks-german| raw:: html + + German version + +.. |website-vvvv| raw:: html + + vvvv + +.. |wiki-osc| raw:: html + + OSC + +.. |website-supercollider| raw:: html + + SuperCollider + +.. |bowelyzer| raw:: html + + bowelyzer + +.. |bowelyzer-about| raw:: html + + about + +.. |website-rpi3| raw:: html + + Raspberry Pi 3s + +.. |website-etherpad| raw:: html + + Etherpad + +.. |wiki-lenovo_thinkpad_x220| raw:: html + + Lenovo Thinkpad X220 + +.. |website-chaos_darmstadt| raw:: html + + Chaos-Darmstadt + +.. |website-trollhoehle| raw:: html + + Trollhöhle + +.. |twitter-trollhoehle| raw:: html + + Twitter + +.. |website-centralstation| raw:: html + + Centralstation + +.. |website-ccc| raw:: html + + CCC + +.. |abbr_json| raw:: html + + JSON + +.. |abbr_lan| raw:: html + + LAN diff --git a/posts/2016/201609-letsencrypt.rst b/posts/2016/201609-letsencrypt.rst new file mode 100644 index 0000000..2ba4221 --- /dev/null +++ b/posts/2016/201609-letsencrypt.rst @@ -0,0 +1,893 @@ +.. title: Let's encrypt it all +.. date: 2016-09-29 20:00 UTC+02:00 +.. modified: 2016-09-30 04:00 UTC+02:00 +.. tags: acme, archlinux, certbot, certificate, dovecot, hidden service, letsencrypt, nginx, openssl, owncloud, postfix, prosody, roundcube, security, ssl, systemd, tls, vpn +.. category: admin +.. slug: lets-encrypt-it-all +.. summary: A short review on half a year of using |website-letsencrypt| for my services. +.. authors: David Runge + +| For a couple of months now I have been using |website-letsencrypt| to generate free and valid certificates for all the services I run. +| In many places the free |wiki-certificate_authority| (short CA) has spread like wild-fire. From small to large scale services, many adopted it and |blog-letsencrypt-1_million_certificates|. +| As a visitor to this website you have probably noticed the small green lock sign next to the address bar. The certificate used for this website is accepted to be valid by your browser (and also by your operating system). +| If you're up for some background knowledge, just read on. If you're up for some hands-on technical stuff, `jump right on to the howto `_. +| Just note: This is a veeeeeeery long article in any case. +| + +Certificate Authority +_____________________ +| *"So, what's so interesting about* |website-letsencrypt| *and why would I want to use it?"* you might ask now. +| To understand the answer to this, you need to learn about the status-quo and how things have been done before |website-letsencrypt| was around. +| +| While anyone can create a certificate and use it for their service, CAs are there to ensure - as a trusted third-party - to your computer, that the certificate, which has been created and is used by you, was really yours to begin with. +| If someone else created a certificate in your name and used it in front of your service (e.g. |wiki-man_in_the_middle|), there would be no way for you to tell, whether you are actually talking to the service you wanted to talk to and the attacker could easily use that certificate to read your traffic (steal your passwords and private data, etc.). +| + +Big players +----------- +| Because of this obvious flaw in the design of how encryption is taking place between computers over a network, CAs have been installed as a way to *validate* your certificate by signing it and thus telling your computer, that the certificate in use is *trusted*, because *they know you*. +| The *they know you* part of it usually meant, that you payed money to a company (CAs up to last year, with the exception of |website-cacert| were private companies), to let it start a script (which i.e. is part of the |website-openssl| software bundle) to sign a certificate or even generate it and afterwards sign it for you. +| Many of them do only casual checking of identity and/or ownership of domains applicated for (e.g. measures such as sending to certain mail addresses on a host are not necessarily secure ways of validating ownership). +| As you can imagine though, this is a money machine. +| + +More Flaws +---------- +| This is the moment, where you should ask yourself: *"Well, what if one of the CAs is a bad boy or messes up in such a way that it issues certificates for anyone without checking or even for services that don't belong to the applicators?"* +| Well... it will come to no surprise: |blog-turktrust_fiasko| and will happen (again). +| In a way this would also be a perfect time to ask yourself, if encrypted traffic is all that secure after all. A little paranoia never hurt. +| +| The certificates with which the CAs sign other certificates and thereby *trust* them are called *root certificates* and are usually shipped by your operating system and/or your web browser. These bundles are used by all applications that understand encrypted traffic to check whether the connection they are about to make is trusted by a CA. +| Now, there is no way of telling whether your CA bundle is sufficient or harmful, but at some point you have to trust your operating system. You do trust your operating system, don't you? Well, |news-microsoft_certificate_blacklist|. +| To circumvent this flaw in turn, Google and Mozilla introduced |wiki-certificate_pinning| in their browsers, which can detect fraudulent certificates for some domains by checking against checksums (derived from a |wiki-hash_function|) of the root certificate with which these domains are signed. These checksums (or sometimes called *hashes*) are shipped with the browser, which should make you wonder, if you can trust those either. +| Sometimes I think, that the more one learns about the topic, the further one wants to move into the woods... +| + +Self-sign +--------- +| For people or associations, that wanted to use their services encrypted, but not go to the lengths of spending a lot of money on top of their hosting costs, it was a typical move to just |wiki-self-signed_certificate| their certificates. I've done it myself and basically, there's no harm in being your own CA. +| Unfortunately (or luckily) browser became more restrictive about whom and how they trusted over the years and made it increasingly painful to use self-signed certificates. In Firefox you could *add an exception* for your certificate and that was that. Chrome/Chromium introduced an annoying red warning page that you had to endure each time you encountered a self-signed certificate. +| + +Teaming up +---------- +| In 2003 |website-cacert| emerged as a community driven project to become a free and non-profit CA. Unfortunately its root certificate was never properly accepted in all operating systems or browsers. +| Although its |wiki-web-of-trust| system for user validation was a neat idea, the association struggled for acceptance due to its insufficient auditing and verification mechanisms for a long time and finally lost to Mozilla's CA certificate policy, which led to a poor |wiki-cacert_inclusion_status| overall. +| + +Encrypt the interwebz +--------------------- +| In 2014 another player entered the stage to make all that pain go away. I was very excited to see a |stream-31c3-letsencrypt| on |website-letsencrypt| amongst the content for |wiki-31c3|. +| Their goal was (and still is) to achieve a maximum throughput in encrypted traffic and for that they were teaming up with some big players (|website-mozilla|, |website-eff|, |website-cisco|, |website-akamai|, |website-university_of_michigan|) to develop their automated validation system, that would offer trusted certificates to anyone running a website for free. +| +| While self-signed and |website-cacert| certificates were becoming increasingly limiting and the sometimes high costs of buying a trusted certificate kept many from encrypting their traffic all together, the services *certified* by the usual CAs on the other hand stood in the light of false pretense to be *more secure* than the formerly named. +| |website-letsencrypt| has indeed come to restore the balance and free what should be free: Encrypted traffic. +| By issuing |wiki-intermediate_certificate|, which were cross-signed by |website-identrust|, |website-letsencrypt| is able to sign and thereby trust certificates it deems valid according to its |github-letsencrypt| validation system. +| Their system is fully automated (implementing |wiki-acme|), easy (and in many ways) to use and doesn't suffer from the above mentioned pitfalls of the usual validation process. +| + +Using letsencrypt +_________________ + +|letsencrypt-howto| + +| The |website-letsencrypt| software bundle is by now packaged for all major Linux distributions (and as a matter of fact, the Internet runs on Linux), so acquiring a valid certificate for your website has become so easy, it's insane. +| Let me tell you about how I do things on |website-archlinux| on this very server. +| Although |website-letsencrypt| lets you freely handle the |wiki-acme| handshake yourself (if you want to do it manually or with |website-letsencrypt_acme_clients|), the community around it wrote a |website-python| based piece of software, that does just that. It's called |website-certbot| . +| While some may argue its backward-compatibility is just dangerous, one could also argue the other way round: Unsecured webservers, running super old |website-python| versions, because their admins can't or won't update, is very dangerous. +| In any case: Every sofware has bugs (some more severe than others), but those are also more likely to get fixed faster, the more people stumble upon them. +| A unifying piece of software such as |website-certbot| is very useful and eases the overall spreading of |website-letsencrypt| . +| Nonetheless, if you're able to do the |wiki-acme| challenge manually and it makes sense in your scenario, you might want to consider that. +| + +certbot +------- +| |website-archlinux| has |website-certbot| in its repositories, so just install the latest version and all of its dependencies: + + .. code:: bash + + pacman -Sy certbot + +| Now would be a good time to have a look at |eff-certbot-nginx-howto| about |website-nginx| in conjunction with |website-certbot|. +| |website-archlinux| of course also has a very |wiki-arch-letsencrypt| on the topic in its wiki. +| At this point I am assuming |website-nginx| is already installed, configured for non-encrypted service and we want to generate certificates for the following domains: **www.domain.tld**, **domain.tld**, **cloud.domain.tld**, **www.cloud.domain.com**, **mail.domain.tld**, **www.mail.domain.tld** (using |wiki-san|). +| Currently the certbot plugin for |website-nginx| is still experimental, so I will refrain from using it and use the webroot method instead. +| + +nginx preparation +----------------- +| Let us have a look at how to configure |website-nginx|, so it will be prepared for the |wiki-acme| challenge. +| + +Snippets +++++++++ + +* we require a directory (*.well-known/acme-challenge/*), that is writable by |website-certbot| (*root*) to place a challenge response on each domain +* the directory must be servable (readable) by |website-nginx| (usually running with the user and group *http*) + +| As the directory can be the same for all the challenges on your server, you can of course just create one and redirect all requests from the outside to it. We will use */srv/http/letsencrypt/* for it and define a configuration block, that we can include anywhere we need it. +| + +* */etc/nginx/letsencrypt-challenge.conf* + + .. code:: nginx + + location ~ /\.well-known/acme-challenge { + root /srv/http/letsencrypt; + default_type "text/plain"; + } + +| This will tell nginx to look inside of */srv/http/letsencrypt* for requests to *./well-known/acme-challenge* on a domain, where we include this. +| +| The following short example is an overview of */etc/nginx/nginx.conf*. Yours might look different and this one is here for demonstrational purposes only! +| Anyhow, I like to separately include the configuration for the different subdomains/domains here, so they will not get mixed up and it will be easier to add or disable functionality. +| + +* */etc/nginx/nginx.conf* + + .. code:: nginx + + worker_processes auto; + error_log /var/log/nginx/error.log; + events { + worker_connections 1024; + } + http { + include mime.types; + default_type application/octet-stream; + gzip on; + sendfile on; + keepalive_requests 55; + keepalive_timeout 55; + # pelican blog + include domain.conf; + # ownCloud + include cloud.domain.conf; + # roundcube mail interface available only through VPN + include mail.domain.conf; + } + +| The initial configuration already shows, that we now have three services that will need to be covered by the certificate, which we want to get. The |website-roundcube| webmail service I picked for demonstrational purposes as a hidden service. This is not meant to badmouth their security, but to show that you can hide your service behind a |abbr_vpn|, if you choose to. +| To achieve something like that, you can use the |website-nginx| geo plugin. When you setup a VPN infrastructure, this will lead to you having a separate connection to your server within a |wiki-private_network|. For the sake of simplicity let us assume your server will have **172.16.0.1** and your client computer **172.16.0.2** as IPs in this setup. +| On your server you can now explicitely look for the correct client and allow or deny access. Another block for the |website-nginx| configuration can be used to let you include this in your domain configurations: +| + +* */etc/nginx/geoblock.conf* + + .. code:: nginx + + geo $is_allowed{ + default 0; + 172.16.0.2 1; + } + +| Here we define a variable called *is_allowed*, which initially defaults to 0. If the request to your server is coming from the IP **172.16.0.2** *is_allowed* will be set to 1. +| **Note**: Add this snippet to your hidden service's configuration file right at the top! +| +| There is one downside to this though, if you choose to have a |website-letsencrypt| certificate for the hidden service: You have to specify an extra check, that excludes calls to *.well-known/acme-challenge* from the geo block and makes it publicly accessible. +| For that to happen you can define another block for multiple inclusion. +| + +* */etc/nginx/letsencrypt-request-check.conf* + + .. code:: nginx + + if ($request_uri ~ \.well-known/acme-challenge) { + set $is_allowed 1; + } + if ($is_allowed = 0){ + return 301 https://domain.tld$request_uri; + } + +| This snippet will set the previously introduced variable *is_allowed* to 1, if the request was correct and will permanently redirect to the main website otherwise. +| As it makes sense to have https enabled on all of your services, the permanent redirect is added to this configuration snippet. You could also separate it out if you like. +| **Note**: You must include *letsencrypt-request-check.conf* **after** *geoblock.conf*, but **before** *letsencrypt-challenge.conf*! +| +| You will have to include the above snippets in your configuration for each of your subdomains/domains and make sure that */srv/http/letsencrypt/* has sufficient permissions. +| This will roughly look as follows: +| + +* */etc/nginx/domain.conf* & */etc/nginx/cloud.domain.conf* + + .. code:: nginx + + server { + listen 80; + listen [::]:80; + # ... + include letsencrypt-challenge.conf; + # ... + } + +| + +* */etc/nginx/mail.domain.conf* + + .. code:: nginx + + include geoblock.conf; + server { + listen 80; + listen [::]:80; + # ... + include letsencrypt-request-check.conf; + include letsencrypt-challenge.conf; + # ... + } + +| + +certbot staging ++++++++++++++++ +| |website-certbot| has a mode called *staging* that basically gets a *"test certificate"* for you, so you can try if everything is working as expected. Sounds safe? Let's do it (as root or with sudo)! + + .. code:: bash + + certbot certonly \ + --staging \ + --agree-tos \ + --renew-by-default \ + --email valid@domain.tld \ + --webroot -w /srv/http/letsencrypt \ + -d domain.tld \ + -d www.domain.tld \ + -d cloud.domain.tld \ + -d www.cloud.domain.tld \ + -d mail.domain.tld \ + -d www.mail.domain.tld + +| All domains are defined seprately using the *-d* flag. The above command will give you an error, if something goes wrong and that usually is quite explicit. +| **Note**: It is very important to test your setup with the staging environment first, because the production environment is rate-limited (and half-baked certs will not do you any good). +| If everything went right, you will now have an intermediate certificate, that in itself is still useless. +| Let's go for the real deal then, shall we? + + .. code:: bash + + certbot certonly \ + --agree-tos \ + --renew-by-default \ + --email valid@domain.tld \ + --webroot -w /srv/http/letsencrypt \ + -d domain.tld \ + -d www.domain.tld \ + -d cloud.domain.tld \ + -d www.cloud.domain.tld \ + -d mail.domain.tld \ + -d www.mail.domain.tld + +| This should return a success message, with the note, that your certificate has been saved to */etc/letsencrypt/live/domain.tld/fullchain.pem* and until when that certificate is valid. +| Congratulations! You just generated a signed certificate, that is valid for the above domains and is recognized by operating systems and browsers! +| + + +Production +---------- +| Before we can include the certificate in the |website-nginx| configuration for each domain though, it is time to think about proper |wiki-ssl_tls| settings (|wiki-cipher_suite|, |wiki-ssl_protocols|, |wiki-dh_params|) and security headers (|mozilla-content_security_policy|, |mozilla-cross_origin_resource_sharing|, |mozilla-http_strict_transport_security|, |mozilla-x_content_type_options|, |mozilla-x_frame_options|, |mozilla-x_xss_protection|). +| Luckily, already a lot of other people have thought about these issues and provided their expertise. Just look at |github-nginx_config|, |blog-ssl_security_on_nginx| or at the |mozilla-ssl_config_generator|. +| + +moar snippets ++++++++++++++ +| To include safe settings for |website-nginx| in all domain configurations, we will create some more snippets and will be happy about this form of reusability! +| + +* */etc/nginx/tls.conf* + + .. code:: nginx + + ssl_certificate /etc/letsencrypt/live/domain.tld/fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live/domain.tld/privkey.pem; + ssl_session_cache shared:SSL:50m; + ssl_session_timeout 1d; + ssl_session_tickets off; + ssl_dhparam /etc/nginx/dhparam.pem; + ssl_protocols TLSv1.2; + ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256'; + ssl_prefer_server_ciphers on; + ssl_stapling on; + ssl_stapling_verify on; + ssl_trusted_certificate /etc/letsencrypt/live/domain.tld/fullchain.pem; + resolver 8.8.8.8; + +| **Note**: I chose a very modern approach towards **ssl_protocols** by enabling only *TLSv1.2* at this point. Depending on your clients, you might want to use *'TLSv1 TLSv1.1 TLSv1.2'* instead. +| To generate the needed *dhparam.pem* (2048 bits recommended) we can use |website-openssl| as root: + + .. code:: bash + + openssl dhparam -out /etc/nginx/dhparam.pem 2048 + +* */etc/nginx/security_headers.conf* + + .. code:: nginx + + add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;"; + add_header X-Content-Type-Options nosniff; + add_header X-Frame-Options "SAMEORIGIN"; + add_header X-XSS-Protection "1; mode=block"; + add_header X-Robots-Tag "none"; + +| A little note on the **Content-Security-Policy** here: Usually one would try to have the targets (**default-src**, **connect-src**, **img-src**, **script-src**, **style-src**) be set to *'self'*. Due to the inline |abbr_css| and Javascript in services such as |website-owncloud| and |website-roundcube|, this is not possible though, so *'unsafe_inline'* and *'unsafe_eval'* have to be added as well for some of them. +| At this point you could of course also choose to create differing *'security_headers'* inclusions for the services you run. +| Depending on which are running, you will want to monitor your developer console in your browser closely after using this security header. It will tell you, if CFP is blocking some resource (and possibly making it unusable). + +domain configurations ++++++++++++++++++++++ + +| Following are the three different configurations for the services (I won't go into detail about |readthedocs-uwsgi| here, but in a coming article I will). + +* */etc/nginx/domain.conf*: + + .. code:: nginx + + # redirect all unencrypted traffic to https + server { + listen 80 default_server; + server_name domain.tld www.domain.tld; + return 301 https://domain.tld$request_uri; + } + + # redirect all traffic to www. to the plain url + server { + listen 443 ssl; + listen [::]:443 ssl; + server_name www.domain.tld; + return 301 https://domain.tld$request_uri; + } + + server { + listen 443 default_server; + listen [::]:443 ssl default_server; + server_name domain.tld; + include tls.conf; + # your pelican blog resides here + root /srv/http/websites/domain.tld; + # make sure to log + access_log /var/log/nginx/access.domain.log; + error_log /var/log/nginx/error.domain.log; + error_page 403 404 /404/index.html; + error_page 500 502 503 504 /50x.html; + # include security headers + include security_headers.conf; + add_header Content-Security-Policy "default-src 'self'; connect-src 'self'; img-src 'self'; script-src 'self'; style-src 'self'"; + # include the letsencrypt snippet + include letsencrypt-challenge.conf; + + location / { + index index.html index.htm; + try_files $uri $uri/ $uri/index.html; + } + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + } + + location = /50x.html { + root /usr/share/nginx/html; + } + } + +| + +* */etc/nginx/cloud.domain.conf* + + .. code:: nginx + + # redirect all unencrypted traffic to https + server { + listen 80; + listen [::]:80; + server_name cloud.domain.tld www.cloud.domain.tld; + return 301 https://cloud.domain.tld$request_uri; + } + + # redirect www. to the plain domain + server { + listen 443 ssl; + listen [::]:443 ssl; + server_name www.cloud.domain.tld; + return 301 https://cloud.domain.tld$request_uri; + } + + server { + listen 443 ssl; + listen [::]:443 ssl; + server_name cloud.domain.tld; + include tls.conf; + error_page 403 /core/templates/403.php; + error_page 404 /core/templates/404.php; + # make sure to log + access_log /var/log/nginx/access.cloud.domain.log; + error_log /var/log/nginx/error.cloud.domain.log; + #this is to avoid Request Entity Too Large error + client_max_body_size 10G; + # include security headers (the rest are set by ownCloud itself already) + add_header Content-Security-Policy "default-src 'self'; connect-src 'self'; img-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline' 'unsafe-eval'"; + add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;"; + # include the letsencrypt snippet + include letsencrypt-challenge.conf; + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + } + + location ~ ^/(?:\.htaccess|data|config|db_structure\.xml|README) { + deny all; + log_not_found off; + access_log off; + } + + location ~ ^(.+\.php)(.*)$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/owncloud.sock; + uwsgi_intercept_errors on; + } + + location / { + root /usr/share/webapps/owncloud; + index index.php; + rewrite ^/.well-known/host-meta /public.php?service=host-meta last; + rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last; + rewrite ^/.well-known/carddav /remote.php/dav/ redirect; + rewrite ^/.well-known/caldav /remote.php/dav/ redirect; + rewrite ^(/core/doc/[^\/]+/)$ $1/index.html; + rewrite ^/caldav(.*)$ /remote.php/dav$1 redirect; + rewrite ^/carddav(.*)$ /remote.php/dav$1 redirect; + rewrite ^/webdav(.*)$ /remote.php/dav$1 redirect; + try_files $uri $uri/ /index.php; + } + + location ~ ^/.(?:jpg|jpeg|gif|bmp|ico|png|css|js|swf)$ { + expires 30d; + access_log off; + } + } + +| + +* */etc/nginx/mail.domain.conf* + + .. code:: nginx + + # include the geoblock snippet + include geoblock.conf; + + # redirect all unencrypted traffic to https + server { + listen 80; + listen [::]:80; + server_name mail.domain.tld www.mail.domain.tld; + return 301 https://mail.domain.tld$request_uri; + } + + # redirect www. to the plain domain + server { + listen 443; + listen [::]:443 ssl; + server_name www.mail.domain.tld; + return 301 https://mail.domain.tld$request_uri; + } + server { + listen 443 ssl; + listen [::]:443 ssl; + server_name mail.domain.tld; + include tls.conf; + # make sure to log + access_log /var/log/nginx/access.mail.domain.log; + error_log /var/log/nginx/error.mail.domain.log; + root /usr/share/webapps/roundcubemail; + #this is to avoid Request Entity Too Large error + client_max_body_size 20M; + # include security headers + include security_headers.conf; + add_header Content-Security-Policy "default-src 'self'; connect-src 'self'; img-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'"; + # include the request-check snippet + include letsencrypt-challenge.conf; + # include the letsencrypt snippet + include letsencrypt-challenge.conf; + + location / { + index index.php; + try_files $uri $uri/$args @roundcubemail; + } + + location @roundcubemail { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/roundcubemail.sock; + } + + location ~ ^/favicon.ico$ { + root /usr/share/webapps/roundcubemail/skins/classic/images; + log_not_found off; + access_log off; + expires max; + } + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + expires 30d; + } + + # Deny serving some files + location ~ ^/(composer\.json-dist|composer\.json|package\.xml|CHANGELOG|INSTALL|LICENSE|README\.md|UPGRADING|bin|config|installer|program\/(include|lib|localization|steps)|SQL|tests)$ { + deny all; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + deny all; + access_log off; + log_not_found off; + } + } + +| As you can see here, you have to exclude *.well-known/acme-challenge/* from denying access to all directories beginning with a dot. +| + +Bringing it up +++++++++++++++ +You should now check your |website-nginx| configuration (as root): + + .. code:: bash + + nginx -t + +| This should tell you if something is wrong. Make sure to fix all problems, else |website-nginx| will not come back up after restarting it! +| If all is well, restart the web server (as root): + + .. code:: bash + + systemctl restart nginx + +| Et voila! Your website should now serve over https! +| You might want to use the |website-mozilla_observatory| now to scan for issues in your setup and to optimize it. +| + +Postfix ++++++++ +Your mail server can also use this certificate now (if your |wiki-mx_record| points to one of the domains the certificate was issued for). + +* */etc/postfix/main.cf* + + .. code:: ini + + smtpd_tls_cert_file = /etc/letsencrypt/live/domain.tld/fullchain.pem + smtpd_tls_key_file = /etc/letsencrypt/live/domain.tld/privkey.pem + +Dovecot ++++++++ +The same counts for your |abbr_imap| server: + +* */etc/dovecot/dovecot.conf* + + .. code:: ini + + ssl_cert = the amount of issued certificates has grown over 1 million in just four months + +.. |website-letsencrypt| raw:: html + + Let's Encrypt + +.. |wiki-certificate_authority| raw:: html + + Certificate Authority + +.. |wiki-man_in_the_middle| raw:: html + + man-in-the-middle attack + +.. |website-cacert| raw:: html + + CAcert + +.. |website-openssl| raw:: html + + OpenSSL + +.. |blog-turktrust_fiasko| raw:: html + + This has happened + +.. |news-microsoft_certificate_blacklist| raw:: html + + that's too bad + +.. |wiki-certificate_pinning| raw:: html + + certificate pinning + +.. |wiki-self-signed_certificate| raw:: html + + self-sign + +.. |wiki-hash_function| raw:: html + + hash function + +.. |wiki-cacert_inclusion_status| raw:: html + + inclusion status + +.. |wiki-web-of-trust| raw:: html + + web-of-trust + +.. |stream-31c3-letsencrypt| raw:: html + + presentation + +.. |wiki-31c3| raw:: html + + 31C3 + +.. |website-mozilla| raw:: html + + Mozilla + +.. |website-eff| raw:: html + + EFF + +.. |website-university_of_michigan| raw:: html + + University of Michigan + +.. |website-cisco| raw:: html + + Cisco Systems + +.. |website-akamai| raw:: html + + Akamai + +.. |wiki-intermediate_certificate| raw:: html + + intermediate certificates + +.. |website-identrust| raw:: html + + IdenTrust + +.. |github-letsencrypt| raw:: html + + openly developed + +.. |website-archlinux| raw:: html + + Arch Linux + +.. |wiki-acme| raw:: html + + ACME + +.. |website-python| raw:: html + + Python + +.. |website-certbot| raw:: html + + certbot + +.. |eff-certbot-nginx-howto| raw:: html + + what the EFF has to tell you + +.. |wiki-arch-letsencrypt| raw:: html + + useful article + +.. |website-nginx| raw:: html + + nginx + +.. |letsencrypt-howto| raw:: html + + + +.. |website-roundcube| raw:: html + + roundcube + +.. |readthedocs-uwsgi| raw:: html + + uWSGI + +.. |wiki-private_network| raw:: html + + private network + +.. |website-letsencrypt_acme_clients| raw:: html + + another client + +.. |wiki-san| raw:: html + + Subject Alternative Name (SAN) + +.. |wiki-cipher_suite| raw:: html + + cipher suite + +.. |wiki-ssl_protocols| raw:: html + + protocols + +.. |wiki-dh_params| raw:: html + + Diffie-Hellman key exchange + +.. |wiki-ssl_tls| raw:: html + + SSL/TLS + +.. |mozilla-content_security_policy| raw:: html + + Content Security Policy (CSP) + +.. |mozilla-cross_origin_resource_sharing| raw:: html + + Cross-origin Resources Sharing (CORS) + +.. |mozilla-http_strict_transport_security| raw:: html + + HTTP Strict Transport Security (HSTS) + +.. |mozilla-x_content_type_options| raw:: html + + X-Content-Type-Options + +.. |mozilla-x_frame_options| raw:: html + + X-Frame-Options (XFO) + +.. |mozilla-x_xss_protection| raw:: html + + X-XSS-Protection + +.. |github-nginx_config| raw:: html + + this + +.. |blog-ssl_security_on_nginx| raw:: html + + this + +.. |mozilla-ssl_config_generator| raw:: html + + Mozilla's SSL config generator for web servers + +.. |website-owncloud| raw:: html + + ownCloud + +.. |website-mozilla_observatory| raw:: html + + Mozilla Observatory + +.. |wiki-mx_record| raw:: html + + MX record + +.. |website-systemd| raw:: html + + systemd + +.. |website-prosody| raw:: html + + prosody + +.. |abbr_vpn| raw:: html + + VPN + +.. |abbr_css| raw:: html + + CSS + +.. |abbr_imap| raw:: html + + IMAP + +.. |abbr_xmpp| raw:: html + + XMPP diff --git a/posts/2016/201610-uwsgi.rst b/posts/2016/201610-uwsgi.rst new file mode 100644 index 0000000..886672b --- /dev/null +++ b/posts/2016/201610-uwsgi.rst @@ -0,0 +1,1468 @@ +.. title: Securely serving webapps using uWSGI +.. date: 2016-10-08 09:00 UTC+02:00 +.. modified: 2016-10-08 05:00 UTC+02:00 +.. tags: application server, archlinux, cgit, mediawiki, nginx, owncloud, php, python, redis, roundcube, security, sockets, systemd, uwsgi, webapps, wordpress +.. category: admin +.. slug: securely-serving-webapps-using-uwsgi +.. summary: An introductory on securely and dynamically serving many webapps with the help of |website-nginx| and |website-uwsgi| with |website-systemd| on |website-archlinux| +.. authors: David Runge + +| Ever since I have been running my own |website-archlinux| box to serve my services, I used |website-nginx| in conjunction with |website-uwsgi|. +| So instead of using |website-php-fpm| and be limited to just |website-php|, I can use a single application server to do all of them (|wiki-cgi|, |website-python|, |website-php| and even the stuff I don't use, such as |website-ruby|, |website-mono|, |website-java|, |website-lua|, |website-perl|, |website-webdav|). They are all separately installable as plugins. +| Static sites, such as this, default to being served by |website-nginx| directly of course. +| Over time I found |website-uwsgi| to be a very versatile and powerful piece of software that has many advantages (over e.g. |website-apache|): + +* socket activation +* webapp encapsulation and jailing +* self-healing +* being able to separetely manage services +* exit after idle + +| I'll explain the services I use (|website-mantisbt|, |website-roundcube|, |website-owncloud|, |website-mailman|, |website-stikked|, |website-wordpress|, |website-postfixadmin|, |website-phpmyadmin|, |website-cgit|, |website-mediawiki|, |website-etherpad| ) along with configuration examples and their possible pitfalls. +| In my last post about `Let's Encrypt <../2016/lets-encrypt-it-all>`_ I already showed some examples on how to configure |website-nginx| for the use with |website-uwsgi|. Let's jump right in. + +.. TEASER_END + +Preparing nginx +_______________ + +| |website-nginx| can serve dynamic websites only indirectly, because it is a web server for static content (|wiki-html|, |wiki-css|, |wiki-javascript|, images, videos, compressed files, etc.), unlike |website-apache|, which uses plugins to take care of many scripting languages (|website-php| and the like). +| In combination with |website-uwsgi| you are able to direct calls to dynamic content to something that handles those best: an application server. Meanwhile |website-nginx| will keep on serving the remaining static content. +| This form of encapsulation has some noticeable security advantages, as every webapp is handled by a separate instance of that application server (and not your web server, which is less likely to blow up in your face because of security flaws in the used scripting language), and that in turn is only accessible through your web server. +| Obviously this also makes it possible to use |website-nginx| as a |wiki-load_balancing| and |wiki-proxy_server|, as you can have one machine serve your domains and just redirect the traffic to other machines plainly serving the webapps. +| I will keep to examples using a single machine (for brevity). +| +| |website-nginx| ships with */etc/nginx/uwsgi_params* holding a set of parameters for the application server, that are set to some of the web server's internally used variables. +| |website-uwsgi| uses a list of modifiers, that are explained in more detail in the list of |doc-uwsgi_packet_descriptions| and of which some correspond to the usage of certain script languages. +| When redirecting to a webapp |website-nginx| uses *uwsgi_pass* in conjunction with the *uwsgi_modifier1* stating the type of application: + + .. code:: nginx + + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/mywebapp.sock; + +| + +Hardening uWSGI +_______________ +| |website-uwsgi| is quite a complex piece of software, but fortunately has a pretty well documented feature set and code base. +| The way it is used in a |website-systemd| context on |website-archlinux| can be improved though (and will hopefully in the future). +| I'm currently only using socket activation for my webapps (not in |doc-uwsgi_emperor_mode|), so all examples will be about how to set that up correctly. +| Following are the current service and socket file shipped with the package in the |website-arch_community_repo|. + +* */usr/lib/systemd/system/uwsgi-secure@.service* + + .. code:: ini + + [Unit] + Description=uWSGI service unit + After=syslog.target + + [Service] + ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini + ExecReload=/bin/kill -HUP $MAINPID + ExecStop=/bin/kill -INT $MAINPID + Restart=always + Type=notify + StandardError=syslog + NotifyAccess=all + KillSignal=SIGQUIT + + [Install] + WantedBy=multi-user.target + +| + +* */usr/lib/systemd/system/uwsgi-secure@.socket* + + .. code:: ini + + [Unit] + Description=Socket for uWSGI %I + + [Socket] + # Change this to your uwsgi application port or unix socket location + ListenStream=/run/uwsgi/%I.sock + + [Install] + WantedBy=sockets.target + +| As the *@* in the service/socket files already suggest: You start them using a configuration file (to be found in */etc/uwsgi/*) name as parameter, similar to: + + .. code:: bash + + systemctl start uwsgi-secure@mywebapp + +| When using socket activation, you would do + + .. code:: bash + + systemctl start uwsgi-secure@mywebapp.socket + +| which will then start the *uwsgi@mywebapp.service* automatically, once the socket is accessed by your web server. +| Starting your webapp in this context generally means, using a configuration file for uwsgi, found in */etc/uwsgi/mywebapp.ini*. +| Let's pretend that *mywebapp* is a |website-PHP| application. This is an abbreviated example of how your configuration might look like: + +* */etc/uwsgi/mywebapp.ini* + + .. code:: ini + + [uwsgi] + # name the process + procname-master = mywebapp + # define the plugin + plugins = php + # define a master process for this app + master = true + # this is where the socket resides + socket = /run/uwsgi/%n.sock + # we want to use this user and group (or any other) + uid = http + gid = http + # give this application a maximum of 10 processes + processes = 10 + + # dynamic scaling + # minimum amount of workers/processes to keep at all times + cheaper = 2 + # increase workers/processes by step + cheaper-step = 1 + + # mark as idle after 10 minutes + idle = 600 + # kill the webapp when it is idle + die-on-idle = true + + # allow no other extenseion than .php + php-allowed-ext = .php + # fix our application in this directory + php-docroot = /usr/share/webapps/mywebapp + # set the standard index + php-index = index.php + php-set = date.timezone=Europe/Berlin + # the application needs access to the following directories + php-set = open_basedir=/tmp/:/usr/share/webapps/mywebapp:/etc/webapps/mywebapp + # this is where we save our sessions + php-set = session.save_path=/tmp + + # mywebapp needs the following PHP extensions + php-set = extension=curl.so + php-set = extension=gd.so + php-set = extension=imagick.so + php-set = extension=intl.so + php-set = extension=mysqli.so + php-set = extension=pdo_mysql.so + +| As you can see: You can (and should) setup your own |website-php| environment for your webapp. All settings will only be available to that specific app (alongside global settings found in */etc/php/php.ini*, */etc/php/conf.d/*). +| My suggestion is to disable all system-wide |website-php| settings and then start to build settings for all your applications. This is much safer, than e.g. an extensive *open_basedir* for all applications. On top: Many applications will not need all the extensions enabled, so just enable the ones they need! +| +| You probably also noticed the *idle* and *die-on-idle* settings here. This will make |website-uwsgi| exit itself, when it is not needed after a given time. This feature will not work with the provided service files however, because |website-systemd| will restart the service automatically (given the above service files). Once |website-uwsgi| is running, it might exit, but will start again immediately, which is not a resource gentle approach at all. +| The application server provides a non-zero, non-one exit code upon exiting by itself. To |website-systemd| this by default means *failure* though. So, how do we fix that and what kind of exit codes does |website-uwsgi| actually give? +| To find out about that, let's dig into *uwsgi.h* in the |website-uwsgi| source code (at the time of writing version *2.0.14*), where we will find the following: + +* *uwsgi-2.0.14/uwsgi.h* + + .. code:: c + + #define UWSGI_RELOAD_CODE 17 + #define UWSGI_END_CODE 30 + #define UWSGI_EXILE_CODE 26 + #define UWSGI_FAILED_APP_CODE 22 + #define UWSGI_DE_HIJACKED_CODE 173 + #define UWSGI_EXCEPTION_CODE 5 + #define UWSGI_QUIET_CODE 29 + #define UWSGI_BRUTAL_RELOAD_CODE 31 + #define UWSGI_GO_CHEAP_CODE 15 + +| As you can see, we have exit code *15*, *17*, *29* and *30* reserved for non-failing exits, while *26* is used in |doc-uwsgi_emperor_mode| and *22*, *5*, *173*, *31* are used for failed exits or even worse. +| |website-systemd| however treats every non-zero exit code in its services as a failure and therefore would not start the service again, once it killed itself and was started by socket activation again afterwards. +| Luckily, the many configuration possibilities of service files come to help. Here is what I came up with (with added hardening): + +* */etc/systemd/system/uwsgi-private@.service* + + .. code:: ini + + [Unit] + Description=uWSGI service unit + After=syslog.target + + [Service] + ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini + Type=notify + SuccessExitStatus=15 17 29 30 + StandardError=syslog + NotifyAccess=all + KillSignal=SIGQUIT + PrivateDevices=yes + PrivateTmp=yes + ProtectSystem=full + ReadWriteDirectories=/etc/webapps /var/lib/ + ProtectHome=yes + NoNewPrivileges=yes + + [Install] + WantedBy=multi-user.target + +| + +* */etc/systemd/system/uwsgi-private@.service* + + .. code:: ini + + [Unit] + Description=Socket for uWSGI %I + + [Socket] + # Change this to your uwsgi application port or unix socket location + ListenStream=/run/uwsgi/%I.sock + + [Install] + WantedBy=sockets.target + +| While the socket file is just a copy of the original, I have tweaked the service. +| This way the above mentioned exit codes are treated as success, instead of failure by |website-systemd| and each |website-uwsgi| instance will get its own private temporary directory below */tmp/*. +| Additionally, the */home*, */root* and */run/user* directories appear empty and system directories, such as */boot*, */usr* and */etc* are read-only to the service. +| Because of configuration and temporary data, I excluded */etc/webapps* and */var/lib* from the above rules. +| For further information on these settings, have a look at the |website-systemd_exec|. +| +| Now a proper starting via socket activation, (harmless) suicide of the service and a re-activation (again via socket) can take place! +| + +Webapps +_______ +| I will go through many examples, that facilitate this setup (some with varying backends though). +| For brevity and due to `my earlier post <../2016/lets-encrypt-it-all>`_ I will only explain whatever happens within |website-nginx|'s server directive (whether you choose to serve your webapps encrypted or not, is not up to me, although I would always encourage encryption!). +| + +MantisBT +-------- +| For a couple of weeks now, I have been maintaining |website-mantisbt| in the |website-aur|, since it was dropped from the |website-arch_community_repo| earlier and I always wanted to try a self-hosted bug tracker. It is a |website-php| based application, that is actively maintained, but ironically also still features many bugs (and then there was that change to |wiki-php7|). +| It is |archwiki-mantisbt|, but once you get the grip on it, it's actually quite nice and has a bunch of interesting features. +| +| Here is, how I serve it on a subdomain + +* */etc/nginx/mantisbt.conf* + + .. code:: nginx + + # ... + + location ~ ^/(admin|core|doc|lang) { + deny all; + access_log off; + log_not_found off; + } + + location / { + index index.php; + try_files $uri $uri/ @mantisbt; + } + + location @mantisbt { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/mantisbt.sock; + } + + location ~ \.php?$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/mantisbt.sock; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| + +* */etc/uwsgi/mantisbt.ini* + + .. code:: ini + + [uwsgi] + procname-master = mantisbt + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 600 + die-on-idle = true + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/mantisbt + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/tmp/:/usr/share/fonts/TTF:/usr/share/webapps/mantisbt:/usr/share/webapps/mantisbt/core:/etc/webapps/mantisbt + php-set = session.save_path=/tmp + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = post_max_size=64M + php-set = upload_max_filesize=64M + php-set = always_populate_raw_post_data=-1 + + php-set = extension=curl.so + php-set = extension=gd.so + php-set = extension=imagick.so + php-set = extension=intl.so + php-set = extension=mysqli.so + php-set = extension=pdo_mysql.so + +| + +Roundcube +--------- +| I have used the excellent webmail frontend for many years now and it just keeps getting better, I think. I would not want to miss its nice features ranging from |wiki-sieve| and |website-gnupg| integration, to password reset and simple folder management. +| + +* */etc/nginx/roundcube.conf* + + .. code:: nginx + + # ... + + location / { + index index.php; + try_files $uri $uri/$args @roundcubemail; + } + + location @roundcubemail { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/roundcubemail.sock; + } + + location ~ ^/favicon.ico$ { + root /usr/share/webapps/roundcubemail/skins/classic/images; + log_not_found off; + access_log off; + expires max; + } + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + expires 30d; + } + + # Deny serving some files + location ~ ^/(composer\.json-dist|composer\.json|package\.xml|CHANGELOG|INSTALL|LICENSE|README\.md|UPGRADING|bin|config|installer|program\/(include|lib|localization|steps)|SQL|tests)$ { + deny all; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + deny all; + access_log off; + log_not_found off; + } + + # ... + +| + +* */etc/uwsgi/roundcubemail.ini* + + .. code:: ini + + [uwsgi] + procname-master = roundcubemail + plugins = php + socket = /run/uwsgi/%n.sock + master = true + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 60 + die-on-idle = true + ; create a cache with 1000 items named roundcube + cache2 = name=roundcube,items=1000 + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/roundcubemail + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = session.save_path=/tmp + php-set = session.save_handler=uwsgi + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = open_basedir=/tmp/:/usr/share/webapps/roundcubemail/:/etc/webapps/roundcubemail/:/var/cache/roundcubemail/:/var/log/roundcubemail/:/secure/location/of/gnupg/keys/for/enigma:/usr/bin/gpg:/usr/bin/gpg-agent + php-set = post_max_size=64M + php-set = upload_max_filesize=64M + php-set = error_reporting=E_ALL + php-set = log_errors=On + php-set = extension=exif.so + php-set = extension=iconv.so + php-set = extension=intl.so + php-set = extension=imap.so + php-set = extension=mcrypt.so + php-set = extension=pdo_mysql.so + php-set = extension=pspell.so + php-set = extension=zip.so + +| + +ownCloud +-------- +| I guess the open-source cloud solution |website-owncloud| has by now reached many homes and work places. It has so many useful features (extended by apps), that it is hard to keep track. +| In any case it is very useful for synchronizing your contacts, calendars and files between many devices and enables you to share files with other |website-owncloud| users or the general public. +| + +* */etc/nginx/owncloud.conf* + + .. code:: nginx + + # ... + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + } + + location ~ ^/(?:\.htaccess|data|config|db_structure\.xml|README) { + deny all; + log_not_found off; + access_log off; + } + + location ~ ^(.+\.php)(.*)$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/owncloud.sock; + uwsgi_intercept_errors on; + } + + location / { + root /usr/share/webapps/owncloud; + index index.php; + rewrite ^/.well-known/host-meta /public.php?service=host-meta last; + rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last; + rewrite ^/.well-known/carddav /remote.php/dav/ redirect; + rewrite ^/.well-known/caldav /remote.php/dav/ redirect; + rewrite ^(/core/doc/[^\/]+/)$ $1/index.html; + rewrite ^/caldav(.*)$ /remote.php/dav$1 redirect; + rewrite ^/carddav(.*)$ /remote.php/dav$1 redirect; + rewrite ^/webdav(.*)$ /remote.php/dav$1 redirect; + try_files $uri $uri/ /index.php; + } + + location ~ ^/.(?:jpg|jpeg|gif|bmp|ico|png|css|js|swf)$ { + expires 30d; + access_log off; + } + + # ... + +| + +* */etc/uwsgi/owncloud.ini* + + .. code:: ini + + [uwsgi] + procname-master = owncloud + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 600 + die-on-idle = true + + owncloud_data_dir = /absolute/path/to/where/your/data/resides + owncloud_writable_apps_dir = /absolute/path/to/writable/apps + chdir = %(owncloud_data_dir) + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/owncloud + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=%(owncloud_data_dir):%(owncloud_writable_apps_dir):/tmp/:/usr/share/webapps/owncloud:/etc/webapps/owncloud:/dev/urandom:/run/redis/redis.sock + php-set = session.save_path=/tmp + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = post_max_size=1000M + php-set = upload_max_filesize=1000M + php-set = always_populate_raw_post_data=-1 + php-set = max_input_time=120 + php-set = max_execution_time=60 + php-set = memory_limit=256M + + php-set = extension=bz2.so + php-set = extension=curl.so + php-set = extension=exif.so + php-set = extension=gd.so + php-set = extension=imagick.so + php-set = extension=intl.so + php-set = extension=gmp.so + php-set = extension=iconv.so + php-set = extension=mcrypt.so + php-set = extension=pdo_mysql.so + php-set = extension=redis.so + php-set = extension=sockets.so + php-set = extension=xmlrpc.so + php-set = extension=xsl.so + php-set = extension=zip.so + + cron = -15 -1 -1 -1 -1 curl --silent https://owncloud.domain.tld/cron.php 1>/dev/null + +| You can see here, that |website-uwsgi| is also able to launch a timed command through its *cron* directive. In this case I am using it to have the call to *cron.php* also be handled by the application server, instead of writing a timer service or using crontab. + +Mailman +------- +| The mailing list software |website-mailman| has been around for ages. The |website-python|-based scripts, templates and |wiki-cgi| frontend are used all around the globe in small to large-scale setups. +| Due to its age and the sometimes very quirky adoptation of the software by several Linux distributions, |website-mailman| has a not so trivial setup (after all you have to connect it to your |abbr_mta| and serve its web-frontend). +| It was slightly annoying to set it up in my case, but eventually it all worked out. +| + +* */etc/nginx/mailman.conf* + + .. code:: nginx + + # ... + + # Send all access to / to uwsgi + location / { + gzip off; + include uwsgi_params; + uwsgi_modifier1 9; + uwsgi_pass unix:/run/uwsgi/mailman.sock; + } + + # Set alias for accessing /icons + location /icons { + alias /usr/lib/mailman/icons; + autoindex on; + } + + # Set alias for accessing /archives + location /archives { + alias /var/lib/mailman/archives/public; + autoindex on; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| You might wonder about the */archives* location at this point. I setup my |website-mailman| instance to serve the archive there, instead of *pipermail*: + +* */etc/mailman/mm_cfg.py* + + .. code:: python + + # ... + + DEFAULT_URL_PATTERN = 'https://%s/' + PUBLIC_ARCHIVE_URL = 'https://%(hostname)s/archives/%(listname)s' + + # ... + +| I am also removing the useless */mailman/cgi-bin/* suffix, because I can. +| + +* */etc/uwsgi/mailman.ini* + + .. code:: ini + + [uwsgi] + procname-master = mailman + master = true + plugins = cgi + socket = /run/uwsgi/%n.sock + processes = 1 + threads = 2 + cheaper-step = 1 + idle = 120 + die-on-idle = true + uid = http + gid = http + cgi = /=/usr/lib/mailman/cgi-bin + cgi-index = listinfo + +| As you can see, the frontend does not require a lot of special treatment at all. +| +| There is one pitfall however, which leads us right back to the above proposed |website-systemd| service file. It does not allow the changing of users or rather acquiring of new privilegdes. +| Unfortunately, that is just what |website-mailman| does... +| + +* */etc/systemd/system/uwsgi@.service* + + .. code:: ini + + [Unit] + Description=uWSGI service unit + After=syslog.target + + [Service] + ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini + Type=notify + SuccessExitStatus=15 17 29 30 + StandardError=syslog + NotifyAccess=all + KillSignal=SIGQUIT + PrivateDevices=yes + PrivateTmp=yes + ProtectSystem=full + ReadWriteDirectories=/etc/webapps + ProtectHome=yes + + [Install] + WantedBy=multi-user.target + +| My proposed fix for this is to leave out *NoNewPrivileges=yes* for now, as ugly as this may seem. |website-mailman| seems to be the only webapp I have encountered so far, that requires this. +| + +Stikked +------- +| The |website-php|-based little webapp |website-stikked| is able to be your own little |website-pastebin| replacement. There are also some nice |abbr_cli|'s around for it. +| + +* */etc/nginx/stikked.conf* + + .. code:: nginx + + # ... + + location / { + index index.php; + try_files $uri $uri/ @stikked; + } + + location @stikked { + rewrite ^/(.*)$ /index.php?/$1 last; + } + + location ~ \.php?$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/stikked.sock; + } + + # Deny serving some directories + location ^~ ^/(application|system)/ { + deny all; + } + + # Serve some static files + location ~* ^.+(favicon.ico|static|robots.txt) { + expires 30d; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| + +* */etc/uwsgi/stikked.ini* + + .. code:: ini + + [uwsgi] + procname-master = stikked + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 120 + die-on-idle = true + cache2 = name=stikked,items=1000 + + php-allowed-ext = .php + php-index = index.php + php-docroot = /usr/share/webapps/Stikked + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/tmp/:/usr/share/webapps/Stikked/:/etc/webapps/stikked/ + php-set = session.save_path=stikked + php-set = session.save_handler=uwsgi + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = extension=gd.so + php-set = extension=mysqli.so + + # cleanup pastes every 5 minutes + cron = -5 -1 -1 -1 -1 curl --silent https://stikked.domain.tld/index.php/cron/stringFromConfig + +| Again, I am using |website-uwsgi|'s cron functionality. This time to call |website-stikked| to make it delete old pastes from time to time. +| + +Wordpress +--------- +| Although I try really hard to get around |website-wordpress| wherever I can by now, it is used by many for their websites and I am also still responsible for a few instances myself. According to its |wiki-wordpress|, the |website-php|-based |abbr_cms| has reached a worldwide coverage of more than 25%. +| That's pretty crazy, considering its |wiki-wordpress_vulnerabilities|. + +* */etc/nginx/wordpress.conf* + + .. code:: nginx + + # ... + + index index.php; + + ## Global restrictions + location = /favicon.ico { + log_not_found off; + access_log off; + } + + location = /robots.txt { + allow all; + log_not_found off; + access_log off; + } + + # Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac). + # Keep logging the requests to parse later (or to pass to firewall utilities such as fail2ban) + location ~ /\. { + deny all; + } + + # Deny access to any files with a .php extension in the uploads directory + # Works in sub-directory installs and also in multisite network + # Keep logging the requests to parse later (or to pass to firewall utilities such as fail2ban) + location ~* /(?:uploads|files)/.*\.php$ { + deny all; + } + + ## WordPress multisite subdirectory rules. + # This order might seem weird - this is attempted to match last if rules below fail. + # http://wiki.nginx.org/HttpCoreModule + location / { + try_files $uri $uri/ /index.php?$args; + } + + # Directives to send expires headers and turn off 404 error logging. + location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ { + expires 24h; + log_not_found off; + } + + # Add trailing slash to */wp-admin requests. + rewrite /wp-admin$ $scheme://$host$uri/ permanent; + + # Directives to send expires headers and turn off 404 error logging. + location ~* ^.+\.(ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|rss|atom|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ { + access_log off; + log_not_found off; + expires max; + } + + # Uncomment one of the lines below for the appropriate caching plugin (if used). + #include global/wordpress-ms-subdir-wp-super-cache.conf; + #include global/wordpress-ms-subdir-w3-total-cache.conf; + + # Rewrite multisite '.../wp-.*' and '.../*.php'. + if (!-e $request_filename) { + rewrite /wp-admin$ $scheme://$host$uri/ permanent; + rewrite ^/[_0-9a-zA-Z-]+(/wp-.*) $1 last; + rewrite ^/[_0-9a-zA-Z-]+(/.*\.php)$ $1 last; + } + + # Pass all .php files on to uwsgi + location ~ \.php$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/wordpress.sock; + } + + ## Errors + # redirect server error pages to the static page /50x.html + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } + + # ... + +| This setup is also ready for |website-wordpress|' |website-wordpress_network|. +| + +* */etc/uwsgi/wordpress.ini* + + .. code:: ini + + [uwsgi] + procname-master = wordpress + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper = 1 + idle = 360 + die-on-idle = true + cache2 = name=wordpress,items=1000 + + php-allowed-ext = .php + php-docroot = /srv/http/websites/domain.tld + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/srv/http/websites/domain.tld:/tmp/:/usr/share/pear/ + php-set = upload_max_filesize=24M + php-set = post_max_filesize=64M + php-set = post_max_size=64M + php-set = session.save_path=/tmp + php-set = session.save_handler=uwsgi + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + + php-set = extension=gd.so + php-set = extension=iconv.so + php-set = extension=mysqli.so + + ; run wp-cron.php job for wordpress every 10 minutes + cron = -10 -1 -1 -1 -1 curl --silent https://domain.tld/wp-cron.php 1>/dev/null + +| Yet again, the calling of *wp-cron.php* is taken care of by |website-uwsgi| directly. +| + +PostfixAdmin +------------ +| When using |website-postfix| as your *MTA* and |website-mariadb| as a backend for your user data, |website-postfixadmin| is a very nice and easy choice to add, delete and change accounts, forwards, etc. for all the domains you run. +| Nevertheless, this is most likely one of those webapps you might want to hide behind a geoblocker and use a |abbr_vpn| to access it. + +* */etc/nginx/postfixadmin.conf* + + .. code:: nginx + + # ... + + location / { + index index.php; + } + + # pass all .php or .php/path urls to uWSGI + location ~ ^(.+\.php)(.*)$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/postfixadmin.sock; + } + + location ~ ^/(config|installer|composer.json-dist|.htaccess|CHANGELOG|INSTALL|LICENSE|README.md|UPGRADING) { + access_log off; + log_not_found off; + deny all; + } + + # Serve some static files + location ~* ^.+(robots.txt) { + allow all; + log_not_found off; + access_log off; + expires 30d; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| + +* */etc/uwsgi/postfixadmin.ini* + + .. code:: ini + + [uwsgi] + procname-master = postfixadmin + master = true + plugins = php + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 120 + die-on-idle = true + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/postfixAdmin + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/tmp/:/usr/share/webapps/postfixAdmin/:/etc/webapps/postfixadmin/:/usr/share/doc/postfixadmin/ + php-set = session.save_path=/tmp + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = extension=mysqli.so + php-set = extension=imap.so + +| As you would suspect, what this application needs, is not much. +| I recommend having a very close look at the configuration file though! +| + +phpMyAdmin +---------- +| If you don't feel like writing |wiki-sql| statements to modify your databases, there is |website-phpmyadmin| available to offer a pretty extensive administrative backend. +| This |website-php| webapp is another one I would not necessarily offer for public access (especially not over plain http). +| + +* */etc/nginx/phpmyadmin.conf* + + .. code:: nginx + + # ... + + client_max_body_size 200M; + + location / { + index index.php; + } + + location ~ ^(.+\.php)(.*)$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/phpmyadmin.sock; + } + + # Serve some static files + location ~* ^.+(print.css|favicon.ico|robots.txt) { + expires 30d; + } + + location ~ ^/(setup|CONTRIBUTING.md|ChangeLog|DCO|LICENSE|README|RELEASE-DATE*|composer.json) { + deny all; + } + + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| + +* */etc/uwsgi/phpmyadmin.ini* + + .. code:: ini + + [uwsgi] + procname-master = phpmyadmin + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 600 + die-on-idle = true + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/phpMyAdmin + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/tmp/:/usr/share/webapps/phpMyAdmin:/etc/webapps/phpmyadmin + php-set = session.save_path=/tmp + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = post_max_size=64M + php-set = upload_max_filesize=64M + php-set = extension=bz2.so + php-set = extension=mysqli.so + php-set = extension=mcrypt.so + php-set = extension=zip.so + +| + +cgit +---- +| The blazingly fast |wiki-cgi| web-interface for |website-git| - the amazing |abbr_vcs| - is a must for everyone self-hosting some repositories. +| |website-cgit| does not require a database, is themeable and very configurable. In conjunction with lightweight access control systems, such as |website-gitosis|, you get a very fast and flexible setup. +| + +* */etc/nginx/cgit.conf* + + .. code:: nginx + + # ... + + location ~* ^.+(cgit.(css|png)|favicon.ico|robots.txt|\.well-known/acme-challenge) { + expires 30d; + } + + location / { + try_files $uri @cgit; + } + + location @cgit { + gzip off; + include uwsgi_params; + uwsgi_modifier1 9; + uwsgi_pass unix:/run/uwsgi/cgit.sock; + } + + location = /50x.html { + root /usr/share/nginx/html; + } + + # ... + +| + +* */etc/uwsg/cgit.ini* + + .. code:: ini + + [uwsgi] + procname-master = cgit + master = true + plugins = cgi + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 1 + threads = 2 + cheaper-step = 1 + idle = 120 + die-on-idle = true + cgi = /usr/lib/cgit/cgit.cgi + +| + +Mediawiki +--------- +| The well-known wiki software |website-mediawiki| is used in a variety of projects and useful in many contexts. +| I use it mainly for personal documentation, but it is of course also a great tool for collaborative knowledge representation (e.g. |website-wikipedia|, |website-archlinux_wiki|) and planning (e.g. |website-32c3_wiki|, |website-lac2016|). +| + +* */etc/nginx/mediawiki.conf* + + .. code:: nginx + + # ... + + location / { + index index.php; + try_files $uri $uri/ @mediawiki; + } + location @mediawiki { + rewrite ^/(.*)$ /index.php?title=$1&$args; + } + location ~ \.php5?$ { + include uwsgi_params; + uwsgi_modifier1 14; + uwsgi_pass unix:/run/uwsgi/mediawiki.sock; + } + location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ { + try_files $uri /index.php; + expires max; + log_not_found off; + } + # Restrictions based on the .htaccess files + location ^~ ^/(cache|includes|maintenance|languages|serialized|tests|images/deleted)/ { + deny all; + } + location ^~ ^/(bin|docs|extensions|includes|maintenance|mw-config|resources|serialized|tests)/ { + internal; + } + location ^~ /images/ { + try_files $uri /index.php; + } + # Deny serving files beginning with a dot, but allow letsencrypt acme-challenge + location ~ /\.(?!well-known/acme-challenge) { + access_log off; + log_not_found off; + deny all; + } + + # ... + +| + +* */etc/uwsgi/mediawiki.ini* + + .. code:: ini + + [uwsgi] + procname-master = mediawiki + plugins = php + master = true + socket = /run/uwsgi/%n.sock + uid = http + gid = http + processes = 10 + cheaper = 2 + cheaper-step = 1 + idle = 360 + die-on-idle = true + cache2 = name=mediawiki,items=1000 + + php-allowed-ext = .php + php-docroot = /usr/share/webapps/mediawiki + php-index = index.php + php-set = date.timezone=Europe/Berlin + php-set = open_basedir=/tmp/:/usr/share/pear/:/usr/share/webapps/mediawiki/:/etc/webapps/mediawiki/:/var/lib/mediawiki/:/usr/bin/ + php-set = include_path=.:/usr/share/pear + php-set = log_errors=On + php-set = display_errors=Off + php-set = error_reporting=E_ALL + php-set = upload_max_filesize=128M + php-set = post_max_filesize=128M + php-set = post_max_size=128M + php-set = session.save_path=/tmp + php-set = session.gc_maxlifetime 21600 + php-set = session.gc_divisor 500 + php-set = session.gc_probability 1 + php-set = extension=gd.so + php-set = extension=iconv.so + php-set = extension=intl.so + php-set = extension=mysqli.so + php-set = extension=redis.so + +| |website-mediawiki| instances need proper |website-mediawiki_combating_spam|, especially, if you want to run them in the wild. +| It is no fun to delete hundreds of spam bot users and pages (been there, done that, good times). +| Make sure to spend some time with your configuration and monitor the wiki instance closely! +| + +Etherpad +-------- +| |website-etherpad| is a beast of its own, because it is a |website-nodejs| application, so it does not require any application server. +| Although it is a very useful tool for collaborative work, I am suspicious of its code base, that builds upon a comparibly young |wiki-javascript| framework with sometimes questionable decision making. +| Anyways, it is served similarly to serving a |website-uwsgi| application: + +* */etc/nginx/etherpad-lite.conf* + + .. code:: nginx + + location ~ ^/p/lac2016(.*) { + include pad.sleepmap-auth-lac2016.conf; + try_files $uri @etherpad-lite; + } + + location / { + try_files $uri @etherpad-lite; + } + + location @etherpad-lite { + proxy_pass http://localhost:9001; + proxy_redirect off; + proxy_buffering on; + proxy_request_buffering on; + proxy_read_timeout 150; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } + +| As you can see, this is a proxy setup for all traffic going towards the *location*, which is then served by the *etherpad-lite.service* listening on port *9001*. +| + +Sidenotes +_________ +| You may have noticed the |website-redis| extension used in some of the webapps. The in-memory data structure store can be used to speed up (common) queries to your application. +| +| The shown *processes*, *cheaper*, *cheaper-step*, *idle* and *die-on-idle*, along with language specific settings (e.g. |website-php|) in the |website-uwsgi| configurations might of course require tweaking, depending on your setup and throughput. +| Not all |website-php| webapps work with *session.save_handler=uwsgi*. +| Make sure to tail your |website-nginx| *access* and *error* logs and follow the journal of any webapp you start using. + + .. code:: bash + + tail -f /var/log/nginx/access.domain.tld.log /var/log/nginx/error.domain.tld.log + journalctl -f -u uwsgi-secure@mywebapp -u uwsgi-secure@mywebapp2 + +| All in all I hope this article will be somewhat helpful in setting up some (or all) of the above mentioned applications within the given framework of tools. +| Enjoy a setup, where your webapps work on demand and you can selectively pull the plug on any of them, without touching your web server. + + +.. |website-letsencrypt| raw:: html + + Let's Encrypt + +.. |website-archlinux| raw:: html + + Arch Linux + +.. |website-python| raw:: html + + Python + +.. |website-nginx| raw:: html + + nginx + +.. |website-roundcube| raw:: html + + roundcube + +.. |website-uwsgi| raw:: html + + uWSGI + +.. |readthedocs-uwsgi| raw:: html + + uWSGI + +.. |website-owncloud| raw:: html + + ownCloud + +.. |website-systemd| raw:: html + + systemd + +.. |website-php-fpm| raw:: html + + php-fpm + +.. |website-php| raw:: html + + PHP + +.. |wiki-cgi| raw:: html + + CGI + +.. |website-ruby| raw:: html + + Ruby Rack + +.. |website-mono| raw:: html + + Mono + +.. |website-java| raw:: html + + Java + +.. |website-lua| raw:: html + + Lua + +.. |website-perl| raw:: html + + Perl + +.. |website-webdav| raw:: html + + WebDAV + +.. |website-apache| raw:: html + + Apache + +.. |website-mailman| raw:: html + + Mailman + +.. |website-stikked| raw:: html + + Stikked + +.. |website-wordpress| raw:: html + + Wordpress + +.. |website-postfixadmin| raw:: html + + Postfixadmin + +.. |website-phpmyadmin| raw:: html + + phpMyAdmin + +.. |website-cgit| raw:: html + + cgit + +.. |website-mediawiki| raw:: html + + MediaWiki + +.. |website-mantisbt| raw:: html + + MantisBT + +.. |wiki-html| raw:: html + + HTML + +.. |wiki-css| raw:: html + + CSS + +.. |wiki-javascript| raw:: html + + JavaScript + +.. |wiki-load_balancing| raw:: html + + load balancer + +.. |wiki-proxy_server| raw:: html + + proxy + +.. |doc-uwsgi_packet_descriptions| raw:: html + + packet descriptions + +.. |website-aur| raw:: html + + AUR + +.. |website-arch_community_repo| raw:: html + + community repository + +.. |wiki-php7| raw:: html + + PHP7 + +.. |archwiki-mantisbt| raw:: html + + non-trivial to setup + +.. |doc-uwsgi_emperor_mode| raw:: html + + Emperor mode + +.. |website-systemd_exec| raw:: html + + systemd.exec manual + +.. |wiki-sieve| raw:: html + + sieve + +.. |website-gnupg| raw:: html + + GnuPG + +.. |website-pastebin| raw:: html + + pastebin + +.. |wiki-wordpress| raw:: html + + Wikipedia article + +.. |wiki-wordpress_vulnerabilities| raw:: html + + long history of vulnerabilities + +.. |website-postfix| raw:: html + + Postfix + +.. |website-mariadb| raw:: html + + MariaDB + +.. |wiki-sql| raw:: html + + SQL + +.. |website-git| raw:: html + + git + +.. |website-gitosis| raw:: html + + gitosis + +.. |website-wikipedia| raw:: html + + Wikipedia + +.. |website-archlinux_wiki| raw:: html + + Arch Linux Wiki + +.. |website-32c3_wiki| raw:: html + + 32C3 + +.. |website-lac2016| raw:: html + + LAC2016 + +.. |website-mediawiki_combating_spam| raw:: html + + spam protection + +.. |website-redis| raw:: html + + redis + +.. |website-etherpad| raw:: html + + Etherpad + +.. |website-nodejs| raw:: html + + NodeJS + +.. |website-wordpress_network| raw:: html + + multisite feature + +.. |abbr_mta| raw:: html + + MTA + +.. |abbr_cli| raw:: html + + cli + +.. |abbr_cms| raw:: html + + CMS + +.. |abbr_vpn| raw:: html + + VPN + +.. |abbr_vcs| raw:: html + + VCS + -- cgit v1.2.3-70-g09d2