Jekyll2019-01-17T23:42:56+00:00https://moonkin.github.io/feed.xmlMoonkin’s blogBlog created for describing my experience within the Outreachy internship.Week 5: Resolving the blocker2019-01-18T00:00:00+00:002019-01-18T00:00:00+00:00https://moonkin.github.io/resolving-the-blocker<center><font size="-1" color="lightgrey">Photo by Pascal Debrunner on Unsplash</font></center>
<p><em>Hi, I am Anastasia, an Outreachy intern working on the project <a href="https://wiki.debian.org/Outreachy/Round16/Projects/DerivativesIntegration">“Improve integration of Debian derivatives with Debian infrastructure and community”</a>. Here are my other posts that haven’t appeared in the feed on time:</em></p>
<ul>
<li><a href="/applying-for-outreachy/">Week 1: Applying for Outreachy</a></li>
<li><a href="/my-first-tasks/">Week 3: My first tasks</a></li>
</ul>
<p>This post is about my work on the subscription feature for Debian derivatives - first of the two main issues to be resolved within my internship. And this week’s topic from the organizers is “Think About Your Audience”, especially newcomers to the community and future Outreachy applicants. So I’ll try to write about the feature keeping the most important details but taking into account that the readers might be unfamiliar with some terms and concepts.</p>
<h2 id="what-problem-i-was-trying-to-solve">What problem I was trying to solve</h2>
<p>As I wrote <a href="/my-first-tasks/">here</a>, around the middle of December the Debian derivatives census was
re-enabled. It means that the census scripts started running on a scheduled basis. Every midnight they
scanned through the derivatives’ <a href="https://wiki.debian.org/Derivatives/Census">wiki pages</a>, retrieved
all the info needed, processed it and formed <a href="http://deriv.debian.net/">files and directories</a> for
future use.</p>
<p>At each stage different kinds of errors could occur. Usually that means the maintainers should be
contacted so that they update wiki pages of their derivatives or fix something in their apt
repositories. Checking error logs, identifying new issues, notifying maintainers about them and keeping
in mind who has already been notified and who has not - all this always was a significant amount of work
which should have been eventually automated (at least partially).</p>
<h2 id="what-exactly-i-needed-to-do">What exactly I needed to do</h2>
<p>By design, the subscription system had nothing in common with the daily spam mailing. Notifications
should have only been sent when new issues appeared, which doesn’t happen too often.</p>
<p>It was decided to create several small logs for each derivative and then merge them into one. The resulting log should have also had comments, like which issues are new and which ones have been fixed since the last log was sent.</p>
<p>So my task could be divided into three parts:</p>
<ul>
<li>create a way for folks to subscribe</li>
<li>write a script joining small logs into separate log for each particular derivative</li>
<li>write a script sending the log to subscribers (if it has to be sent)</li>
</ul>
<p>The directory with the small logs was supposed to be kept until the next run to be compared with the new small logs. If some new issues appeared, the combined log would be sent to subscribers; otherwise it would simply be stored in the derivative’s directory.</p>
<h2 id="overcoming-difficulties">Overcoming difficulties</h2>
<p>I ran into first difficulties almost immediately after my changes were deployed. I forgot about the
parallel execution of <code class="highlighter-rouge">make</code>: in the project’s crontab file we use the <code class="highlighter-rouge">make --jobs=3</code>, which means three parallel Make jobs are creating files and directories concurrently. So all can turn to chaos if you weren’t extra careful with the makefiles.</p>
<p>Every time I needed to fix something, Paul had to run a cron job and create most of the files from zero. And for nearly two days we’ve pulled together, coordinating actions through IRC. At the first day’s end the main problem seemed to be solved and I let myself rejoice… and then things went wrong. I was at my wits’ end and felt extremely tired, so I went off to bed. The next day I found the solution seconds after waking up!</p>
<h2 id="cool-things">Cool things</h2>
<p>When finally all went without a hitch, it was a huge relief to me. And I am happy that some folks have already signed up for the errors, and hope they consider the feature useful.</p>
<p>Also I was truly pleased to know that Paul had first learned about makefiles’ order-only prerequisites from my code! That was quite a surprise :)</p>
<h2 id="more-details">More details</h2>
<p>If you are curious, in the <a href="/projects/">Projects</a> section a more detailed report will appear soon.</p>A few words about the subscription feature.Week 3: My first tasks2018-12-25T00:00:00+00:002018-12-25T00:00:00+00:00https://moonkin.github.io/my-first-tasks<center><font size="-1" color="lightgrey">Photo by Raphael Koh on Unsplash</font></center>
<p>Hi! It has now been three weeks since my internship in Debian started. In my <a href="/applying-for-outreachy/">first post</a> I said a few words about the project I am working on, but I’d like to list the key points again.</p>
<h2 id="key-points">Key points</h2>
<p>The central part of the project is the <a href="https://wiki.debian.org/Derivatives/Census">census</a> collecting detailed information about Debian derivatives (software distributions based on Debian). Derivatives may be quite different depending on which Debian release they are based on and how they modify or extend it. Each derivative has its own wiki page where its maintainer is expected to keep the most up-to-date information, like links to the official website, wiki, blogs, forums, git (or other VCS) repositories and mirrors, as well as contact information and apt sources.list snippets.</p>
<p>The <a href="https://salsa.debian.org/deriv-team/census/tree/master/bin">census scripts</a> take information from the wiki pages and process it for integration into various parts of the Debian infrastructure. For example, they find a link to the derivative’s blog, download its web pages, use automatic feed discovery algorithms to find a link to the blog’s Atom/RSS feed and then produce a configuration snippet to be added to the <a href="https://salsa.debian.org/planet-team/config/blob/master/config/config.ini.deriv">aggregated one</a> in the Planet Debian repository. Once it is added, the post from the blog will appear in the <a href="https://planet.debian.org/deriv/">Planet Debian Derivatives</a> feed.</p>
<p>Some of the derivatives in the census are marked as inactive as they are no longer maintained for some reasons. Their information is still present in the wiki but is not processed by the census scripts, so it is mostly preservation of history. And if you are wondering why some distro is inactive, on the wiki page there is often a link to a <a href="https://lists.debian.org/debian-derivatives">discussion with its maintainer</a> or to some other source where the reason is explained.</p>
<p>Directories and files produced by the census scripts are listed <a href="http://deriv.debian.net/">here</a>. If the scripts fail to download or process some data or some other issue occurs, there often are <a href="https://wiki.debian.org/Derivatives/CensusQA">mail templates</a> to be sent to the maintainers. For now there is no option to send notifications automatically, but it will appear in the nearest future as it is the first of the two <a href="https://wiki.debian.org/Outreachy/Round16/Projects/DerivativesIntegration">blockers (main integration tasks)</a>. I am already working on the subscription system and hope to finish it by the New Year :)</p>
<h2 id="my-work">My work</h2>
<p>During the last three weeks I have been working on several small tasks:</p>
<ul>
<li>some that helped to re-enable the census (make its scripts produce <a href="http://deriv.debian.net/">data</a> daily):
<ul>
<li>implementing an <a href="https://salsa.debian.org/deriv-team/census/commit/2cee3414d48fda292f33864999550264537f4887">option</a> to disable some scripts</li>
<li>adding my email to the crontab files to receive the output of the scripts</li>
</ul>
</li>
<li>some connected to Planet Debian Derivatives:
<ul>
<li><a href="https://salsa.debian.org/planet-team/config/commit/9c4c2a873d3a40721624bc8c2e987576ca4f50bd">updating</a> logos and feeds of some derivatives</li>
<li><a href="https://salsa.debian.org/deriv-team/census/commit/359dc268a5d9ebc83bf7647aa0d4094738098f1e">creating</a> a tarball of logos on the server’s side to be downloaded by the <a href="https://salsa.debian.org/deriv-team/census/blob/master/bin/review-planet-config-heads">review script</a></li>
</ul>
</li>
<li>preparations for the future work on the subscription system:
<ul>
<li>synchronizing the census output produced by three parallel jobs and thus having errors listed quite randomly</li>
<li><a href="https://salsa.debian.org/deriv-team/census/commit/09110bb3b203f50101218a857e3bd096981ca620">removing</a> some errors from the output</li>
</ul>
</li>
</ul>
<h2 id="everybody-struggles">Everybody struggles</h2>
<p>Every two weeks interns receive a mail from Outreachy organizers with guidelines and goals for the next couple of weeks. There also are some topics that we are suggested to talk about in our blogs to help future applicants and interns. This week’s topic is “Everybody struggles”, about getting stuck and asking for help. (All my future posts will probably have the how-I-got-stuck section too, because it happens all the time!)</p>
<p>For me getting stuck is as much a part of the learning process as reading, making notes, trying new tools and approaches, searching for the code samples and so on. I only feel foolish when I realize I had misunderstood something at the very start and was moving in the wrong direction. But I try to remind myself that even then it is not a waste of time, as I learn something anyway.</p>
<p class="notice"><strong>For future applicants and interns</strong> You can’t escape being stuck while working on a complex project. And after focusing intensively on some task and still not moving any closer to the solution you will probably feel frustrated. But all this doesn’t just drain your energy, it also shapes your understanding of the task and helps you find useful tools for your future work. So don’t regret the time spent! And if you feel confused and question your chosen approach, mentors are always here to guide you.</p>
<p>In my post about the application process I wrote about some difficulties I had faced while making my first contributions. At that time I was hesitant to ask for help and scared of making mistakes. Also being a non-native English speaker I felt more secure when contacting the project’s mentors by email, so it often took more time to get help than if I used instant messaging<sup>1</sup>. Later I slowly began talking in IRC and even got used to discussing my code publicly. There was no pressure to do so, only encouragement, and I just decided to try one day and all went good. I started receiving help almost immediately and although I continue making mistakes from time to time, I am not as upset about them as in the beginning.</p>
<p class="notice"><strong>For future applicants and interns</strong> After all you aren’t supposed to know everything! You are here to learn. Good luck to you all!</p>
<p><sup>1</sup> <em>BTW, I really struggled to find an IRC client for my Android phone until I came across <a href="https://matrix.org/docs/projects/client/riot-android.html">Riot Matrix client</a>. It is super cool. It integrates several IRC networks like Freenode and OFTC and can work as an IRC bouncer (you remain connected and don’t lose message history). There also are <a href="https://matrix.org/docs/projects/client/riot.html">web</a> and <a href="https://riot.im/desktop.html">desktop</a> versions. Hope it will be of use to someone.</em></p>The work I've done during the first three weeks of the internship and my experience of getting stuck and asking for help.Week 1: Applying for Outreachy2018-12-02T00:00:00+00:002018-12-02T00:00:00+00:00https://moonkin.github.io/applying-for-outreachy<center><font size="-1" color="lightgrey">Photo by LEMUR on Unsplash</font></center>
<h2 id="hi-">Hi! :)</h2>
<p>I am Anastasia, an Outreachy participant, and in this blog I’ll be writing about
my work on the project <a href="https://wiki.debian.org/Outreachy/Round16/Projects/DerivativesIntegration">“Improve integration of Debian derivatives with Debian infrastructure and community”</a>.</p>
<p>Last year I received my undergraduate degree in Computer Science, but I have neither
worked as a programmer, nor contributed to Free and Open-Source Software ever before.
Thanks to Outreachy I started exploring the FOSS world and learned lots of new
things already during the application process.</p>
<p>I first heard about Outreachy in September, just a few days before the applications
opened, so I had to make up my mind rather quickly. Coincidentally, I’ve already
planned quitting my non-tech job until winter and the prospect of searching for a tech
one without practically no real-world experience worried me a lot. A paid, remote and
full-time internship looked exactly what I needed.</p>
<p>Trying to decide I’ve read some former interns’ blogs and all of them said that even if you
are not accepted as an intern, participation in the application process itself can be extremely
fruitful. And now I can say that it’s true!</p>
<h2 id="application-process">Application process</h2>
<p>At first I answered some questions about my background in the <a href="https://www.outreachy.org/apply/eligibility/">initial application</a>.
The next step was picking a project from the list.</p>
<p class="notice"><strong>For future applicants:</strong> All projects have detailed descriptions,
lists of required skills, mentors’ contact info and mentorship styles.</p>
<p>As I had just completed the <a href="https://www.coursera.org/specializations/python">Python specialization</a>
from the University of Michigan and was really determined to learn Python further, I looked through
the projects related to Python programming and then narrowed the list down to a few ones.</p>
<p>I ended up choosing Debian project as it seemed interesting and required some knowledge of shell
scripting which I wanted to improve as well as Python. Also there was Git marked with a <code class="highlighter-rouge">1</code> - it means
that you can be a complete beginner and mentors are ready to teach you. I must say that my Git
skills were very limited. I finished an online course and understood how Git works in
theory, but I haven’t had enough practice because I’ve only tried local repositories. Also I have
never even seen makefiles and was unfamiliar with a lot of Debian-specific stuff, but thought it
would be great to get to know some.</p>
<p>So in the following weeks I had to ask one of the project’s mentors, Paul Wise, several really
newbie questions. His replies were friendly and patient, he often recommended me relevant commands
and explained me things when I got stuck (sometimes repeatedly). Thus by the end of October I gained
more understading and confidence and tried lots of cool things. Thank you, pabs!</p>
<h2 id="my-first-task">My first task</h2>
<p class="notice"><strong>For future applicants:</strong> Your first contribution isn’t supposed to be complex. Just pick something small from the list of application tasks and contact your project’s mentors to introduce yourself and ask for any details!</p>
<p>My first task was a social one and involved no coding. I needed to send an invitation to Debian derivative
not yet present in the <a href="https://wiki.debian.org/Derivatives/Census">census</a>. I used the premade invitation
template, picked two derivatives from the list provided by Paul and send invitations to their maintainers.
Although the task looks simple, it had social importance and also it was important for me as
I’ve learned how mailing lists work :)</p>
<h2 id="later-contributions">Later contributions</h2>
<p>During the <a href="https://www.outreachy.org/apply/make-contributions/">contribution period</a> I worked
on several other social, wiki maintenance and programming tasks. All these tasks are connected.
The information in the wiki needs to be up-to-date as it is processed by the census scripts, and
you need to notify the maintainers about all the issues those scripts detect.</p>
<p>At first I configured Debian Stretch VM and forked the project’s codebase (actually, I cloned
it and created a fork only after I had trouble opening a merge request… remember, limited Git
skills). I didn’t know which options to specify for the <code class="highlighter-rouge">make</code> command, so I was just sitting there
watching while it was creating directories for all the derivatives and producing some cryptic output.
Later I identified several kinds of errors in the output. The easiest ones were about broken
links and missing logo images, and those might have been fixed simply by updating the wiki pages.
So I did it. Some errors stopped appearing and I was proud of myself as if it was the project’s most
important FIXME :)</p>
<p>Speaking of FIXMEs, later I picked some of them too. The files and directories produced by the scripts
are available <a href="http://deriv.debian.net/">here</a>, but there was no Description column before. My first
coding task was to create it. After finishing my work I knew much more about running and configuring
Apache servers, <code class="highlighter-rouge">mod_autoindex</code> module, <code class="highlighter-rouge">.htaccess</code> files and the census codebase itself, as
I had to read the README file thoroughly.</p>
<p>Other FIXME that I’ve chosen was <a href="https://salsa.debian.org/deriv-team/census/commit/01668e1f52b7362e6e3f71bd35655b11a4d449ec">“add dd-lists of patched packages”</a>.
A dd-list is a list of packages and their maintainers and uploaders. To be useful this information must
be the most up-to-date, in other words collected from Debian experimental and unstable suites.</p>
<p>Working on this task I got confused more often than ever before. Final commit looks good, but I can’t
recall how many commits there originally were and I am afraid to look at the corresponding merge request.
Let’s just say there were plenty of them.</p>
<p><strong>What went wrong:</strong></p>
<ul>
<li>I received lots of helpful information from Paul, but just couldn’t handle it properly while misunderstanding
the very purpose of the task and being unable to ask the right questions</li>
<li>I didn’t realise that we needed a <code class="highlighter-rouge">debian.apt</code> directory with Debian sources files, and tried to use
sources from the derivatives <code class="highlighter-rouge">apt</code> directory (some of them even don’t have one)</li>
<li>I spent a huge amount of time trying to understand how to handle the <code class="highlighter-rouge">APT_CONFIG</code> environment
variable inside the makefiles and my script</li>
</ul>
<p><strong>Good moments:</strong></p>
<ul>
<li>As a result I learned how to:
<ul>
<li>write proper commit messages</li>
<li>squash commits</li>
<li>consciously add new targets with prerequisites and rules with automatic variables (all those <code class="highlighter-rouge">$@</code> and
<code class="highlighter-rouge">$<</code> things) to makefiles</li>
<li>use shellcheck tools to correct shell scripts</li>
<li>handle stress and be patient to myself, at least sometimes</li>
</ul>
</li>
<li>In the end the <code class="highlighter-rouge">APT_CONFIG</code> was set correctly!</li>
<li>I proposed some ideas and they were accepted</li>
<li>After all the struggle the <code class="highlighter-rouge">sed</code> syntax seemed relatively clear to me :’)</li>
</ul>
<p>Now I don’t feel so bad about how I had described this contribution on the Outreachy website :)
After reading my description there one would think it all went without a hitch. No, it didn’t!
It was a painful, but also exciting experience, and when all finally clarified it felt like a blessing.
And it’s really cool when your code is reviewed by someone knowledgeable. And also I began to really like makefiles!</p>
<p><strong>What I would advise my past self</strong></p>
<ul>
<li>Don’t panic</li>
<li>Read the README</li>
<li>Try IRC and Matrix, not only emails</li>
<li>Make notes of the useful commands and working scripts, you might need them later</li>
<li>Try to sleep better</li>
</ul>
<h2 id="final-application">Final application</h2>
<p>I had to write about my experience and provide the approximate timeline for the internship period.
I don’t know if the application is supposed to be this long or not, but I wrote about my contributions,
the help I received and my feelings of the whole application process. Also there was a question
about personal or class projects which helped me to gain required skills.</p>
<p class="notice"><strong>For future applicants:</strong> I think it’s good to mention all you’ve done to gain some required skill, be it reading, completing online-courses, solving puzzles, testing code samples or configuring environments. Unfinished projects seem to be good too! I myself wrote about one.</p>
<h2 id="results">Results</h2>
<p>My project had an extended deadline and I needed to wait only ten days until the results were announced.
I thought I would not be selected, but it happened! I remember feeling stunned for a few following days.
I was determined to do my very best, but a bit scared too. But the more I thought about the future, the more I liked it :)
Later in November I quited the job exactly as I had planned and was truly happy that I didn’t have to
search for a new one in a hurry. Because I’ve already had something cool to do!</p>
<p>I want to say thanks to all Outreachy organizers for this awesome opportunity. And good luck to future applicants!
This winter promises to be a great and busy time. Can’t wait to get started!</p>How I became an Outreachy intern in Debian: details of the application process.