Stories
Slash Boxes
Comments

Dev.SN ♥ developers

posted by NCommander on Monday March 24 2014, @03:35PM   Printer-friendly
from the the-much-belated-story-post dept.
It's been a hectic couple of weeks since SoylentNews went public, and our young community has already seen it fair share of difficulties, drama, and growing pains. Due to an absolutely hectic travel schedule combined with good helping of emergencies, and other calamities, I've been dealing with both trying to keep us operational, and working towards defining who we are and what we will be. With my travel on hold until May, and with most of the pressing issues resolving, I feel that we've finally reached a moment where we can stop reacting, and starting planning. With that said, I'd like to take a moment to sit down, talk about recent events, and give you what I see our future as.

Here is what I'm going into on this post:
- My Vision for SoylentNews
- Incorporating: How, and Where
- The Community Voting System and Voting For Our New Name
- Disclosure on Recent Security Issues

NCommander adds: Well, that was bad timing. Our submission drought ended about the time this went live so its already off the main page. I feel that its important that most people read this so they can at least see the call for volunteers so I'm bumping it back to the top of the main page. At some point we need a featured article feature so things like important site news can stay "stuck" to the top.

My Vision for SoylentNews

So, the first, and perhaps most important question is, what are we doing on defining who we are, and how we operate. I'd be lying if I said the changeover in leadership was smooth, but as with any new fledged organization, we're victim to growing pains. There's been one sticking issue on my TODO list though since I took over, and that's provide indication of where we're going and how we're going to get there.

As most of you know, I was in Asia for two weeks due to my corporate job. During that time, I worked out a basic plan of where we are going to go. Not long after I returned from Asia, I wrote an email to the staff mailing list to act as a guiding statement, in lieu of a full manifesto. Unfortunately, my writing time has been curtailed by work, and other day-to-day Soylent operations, so as a down payment on that manifesto, I'd like to share that email publicly. It has been slightly edited for clarity, and to correct some misconceptions from the original. Contextual edits are marked in brackets.

Hey all,

So after two long weeks, I'm finally back home, resting and recovering from a ton of jetlag. I've caught up with recent events, and am finally able to take the time to lay out a clear vision for the future of the site, which is something that I know you have been promised now for some time.

First, I would like to formally announce that we will be organized as a nonprofit ([to become the equivalent] section 501(c) of U.S. tax law). I can hear you thinking, "why a nonprofit, and isn't this something the community should decide? These are valid and excellent questions, and I will address them one by one:

First, the nonprofit thing. To be very blunt, this is a matter of personal values, and commitment to the community. Slashdot sold out relatively early in its life, and changed from owner to owner until CmdrTaco resigned. While this process took over a decade, I consider that to be an exception to the rule. Plenty of other sites have been bought, and then quickly destroyed by greedy PHBs. While, given our current user base, I realize that as a for-profit business we could become rich and then sell out, it would be a betrayal of everything we forked from Slashdot for, a betrayal of the community. I'm not someone to take the money and run; I'm here because I believe we can change the world, and I stand by my word.

A nonprofit organization exists to further its mission, and frankly, I'm unaware (at least in the United States) of any major news source beside Wikinews which isn't for-profit. If we became a for-profit business, its possible (perhaps likely) that sooner or later, someone would offer the board of directors a *shit ton* of money, and to be honest, it is difficult for anyone to say no when a seven digit paycheck is staring them in the face. Slashdot, Digg, SourceForge, and many other sites have been bought out, and subsequently dismantled with the goal of profit and making money. Reddit is an anomaly having been bought out, but remaining largely independent of its mother ship. Furthermore, as a nonprofit, we won't have a corporate overlord that we have to please. We can report the truth, the whole truth, and nothing but the truth. I realize that perhaps its passe, but I'm serious when I say we are going to change the world.

As we become more and more established, I'd like to transition to a world where we can be both an aggregator of news, *and* a primary news source. We can hire full-time editors, and find people to help train our editors, both volunteer and paid, to bring us up to standards on par with ArsTechnica, Engadget, and other large names in this field.

This won't happen overnight. Here's my tentative list of where I want us to be in one year; these goals may seem a bit modest but I would like our goals to be reasonable, and then have us exceed them if possible:

  • [To become the equivalent of] a federally approved (tax-exempt) 501(c) nonprofit organization
  • I expect this to take anywhere between 3-6 months, unlike for-profits, the registration process for these is long and arduous
  • Will require the creation of bylaws, board of trustees, etc.
  • All site assets will be transferred to this once we're founded
  • Selection of a permanent name (within next few weeks, but listing here for completion)
  • SoylentNews Manifesto to provide detail on how we will organize ourselves to strengthen the site and keep it running smoothly
  • SoylentNews Governance Model (see below)
  • Sufficient income to cover server hosting expenses
  • 5M pageviews daily by slash's internal count (slash reports ~2M pageviews daily at this point)
  • Average 30-40 comments per article
  • Sufficient income to support self-hosted servers vs. Linode (dedicated hardware/racks vs. Linode)
  • Sufficient income to allow us to hire at least 2-3 people *if necessary*

Because nonprofits need to jump through a lot of hoops and are subject to a lot of laws and scrutiny, the governance model for the nonprofit itself will be fairly traditional, with a board of directors, written bylaws, etc. That being said, for the most part, Soylent will not be directly affected by this, which acts as a nice segue to my next points.

You may be questioning how I can definitively say that this is something we're doing without polling the community. The answer here is that it falls in the category of an implementation detail. We are subservient to the community, just as a fire hall is subservient to its community. The community isn't directly affected by us being for-profit or nonprofit, they just want a site that's better than the other site, and no risk that we're going to Beta them (as well as being free of the slant).

For the most part, the relationship between the nonprofit and the "SoylentNews" site itself will leave the site relatively free to set its own destiny. My plan for this relationship will be similar to that between the Wikimedia Foundation and Wikipedia, or SPI and Debian, or the original relationship between the federal government and the states in the United States. I'd like to see a future where there are host of Soylent-like sites on other topics, not just technical ones. The most popular articles on Slashdot were those involving politics, but a lot argued that they were rather off-topic for the site. Instead of trying to expand SN's mission, I'd rather see us have a "U.S. Current Events" site or similar, and perhaps a network of interconnected sites under a common constitution which outlines the rights and responsibilities each site has.

Aside from matters of law, each site will be free to mostly run and govern itself. While perhaps it is wishful thinking that we'll have more than just Soylent, there needs to be a clear separation of where the nonprofit ends, and where Soylent begins, lest we end up like for-profit projects where the community takes a back-seat to business needs. By having a defined relationship between the two at the get-go, the grounds on which an elected or appointed board of trustees can interfere with a community-governed site will be strictly limited. What we need is our "Freedom of the Press and Associated Rights" constitution, which will form the basis of the founding bylaws of the nonprofit, and then from there, work on creating a governance model for Soylent with the community, in which Soylent is represented and shielded by the nonprofit in matters of law, business, finance, and the like.

That being said, it has become clear that we need to become incorporated and organized as soon as possible so that we can legally represent ourselves in a sane and viable manner, as well as have an organization to shield individual people from specific prosecution should we ever manage to tick anyone else off. While this has always been the plan, it has become an absolute immediate priority lest another crisis come and make our lives miserable. To this end, Matt [will] post a journal article detailing the legal steps that we will take in the near term to set up the nonprofit.

--NCommander

I realize a lot of you probably wanted something more concrete, but we've been hashing out a lot of things to get to this point. When I took over, I promised the staff and the community that we would have transparency, and I intend to honour that promise for as long as I am in charge. I've said it before, and I will say it again, we are subservient to the community, and serve to fill the needs of that community.

Incorporating: How, and Where

This brings me to my next point. It's been raised that there are concerns with us being (legally) based in the United States. Given recent revelations, its fair to said many of these concerns are valid, and should be addressed. However, incorporation (especially as a non-profit) is difficult, and frequently requires local residency, fluency in the local language, and a strong understanding of local legal system. Furthermore, it is impossible for me to go through the legal codes of every country, determine fact from fiction, and comparatively weigh pros and balances.

I am not against the concept of international incorporation, and the option will remain open for the future. The problem here is specifically initial incorporation. Furthermore, despite everything, I do feeling the United States still has some of the strongest protections for bloggers, journalists, and freedom of press. I'm aware this is an issue that many feel very strongly about, so if you're interested in seeing us incorporate outside the United States, then we need you to step forward, and make yourself known. Come find me on IRC, and we'll go into depth on what is required, and what is expected.

For my part, I'm going to write up a rather in-depth pros and cons going through various case law, business regulations, and such to determine what we get for incorporate within the United States. I'm working with Matt to get a definitive list of questions we need answers to know to seriously consider for any given locale. Incorporation as a non-for-profit is a serious matter, and requires commitment and dedication to see the process to the end. If you're willing to put in the hours, accept any legal responsibilities required, and act as a definitive guru, then you're welcome to step up and make your case for your country.

Do not volunteer for this lightly! I'm going write an extremely detailed dissertation on United States incorporation, likely to be at least 10,000 words long. I expect the same of anyone else who has the commitment and drive to see this through. It will contain answers to the 'important considerations' such as citizenship requirements, legal reporting on matters of finance, as well as summarization (with citations) of journalist protections, relevant case law, and the like, both positive and negative. By taking on this responsibility, you are willing to essentially take charge on the bureaucratic aspects of our legal foundation.

The call for volunteers shall remain open one (1) week from the posting of this article. If no one steps forth to take on the responsibility, we will incorporate in the United States by default. We need to get incorporated both for the legal protections it provides, and to start building sources of funding so this isn't something that can be held up for months with endless discussion. I hope to have a report put together on the United States (with opinions of select states) within a week or two, followed by a discussion period should any viable alternatives step forward.

The Community Voting System and Voting For Our New Name

With that covered, I'd like to move onto talking about our progress on fulfilling our promise to hold a public vote on the site name. We're now over a month since go-live, and with each passing day SoylentNews as a name becomes more and more entrenched. I would have already liked to have the vote, and renamed the site, but as it stands, we're not just there yet. The biggest hold up is we don't have a realistic way to vote on issues; Poll Booth is unacceptable for this role.

Now, there are a million and one online survey sites which we could use, as well as various methods of polling packages. We could use one of these to get the job done quickly, but this is a case where its more important to get it right. We're going to have a future where a fair number of issues will be voted on by the community to be implemented by the staff. Furthermore, we value the privacy of our users. You may have noticed that we don't use anything like Google Analytic or the like on this site, nor do we log IPs of visitors. The only information we collect is a IPLD (MD5SUM salted hash) of the IP, a user name, salted password, and an email address (plus whatever a user enters in their profile).

We can't be compelled to hand over what we don't have, and such any voting infrastructure needs to be something we control, and something the community can audit. I won't pretend that we will have a perfect system the time around, but several devs have been hard at work at building an email based voting system, as well as looking into seeing if we can modify the Polling Booth to be acceptable for such votes. I'll allow our devs to speak for themselves, but I'm hoping we can demo the voting system, and get the ball rolling on the vote this week.

Disclosure on Recent Security Issues

This unfortunately brings me to a less happy subject. We've recently received what we consider a creditable threat against the site, with a supposed vulnerable in slash that will allow someone to own the site. Now, we knew going into this that security was always going to be a concern, especially as we're still tied to Apache 1.3. With threats being made, it was time to, as they say, step up our A game, and go through all the entire backend, make sure that everything is reasonably documented. A complete overview of the most recent round of updates can be found in my journal

Most of our backend infrastructure was put together rather hastily as we went towards go-live with rather little documentation, and hadn't been audited since the initial startup. One thing we found was that it was possible to log into the production machine with an easily guessed username and password which was left open to the world. The account (slash) was non-root nor sudo access, but did have read access to the configuration files that drive slashcode, including the database credentials. I've gone through the auth.log and doesn't appear that this was ever discovered, and we're reasonably sure that no one ever got into the production boxes in this manner. We've rectified the mistake, and implemented strong SSH usage policies to prevent this from reoccurring (see my journal for full details on the new policies). We became aware of this misconfiguration on Thursday shortly after we came back from our scheduled down time, and the mistake was immediately rectified. As we don't believe we were compromised, I held off on public disclosure until we finished auditing and hardening lest it service as an invitation to hit us while we were down. As this is our first known security issue, feedback on our disclosure practices is welcome.

As part of this audit, we've established secure ways to access our nodes, a list of all hardware and what they're running, and a set of directions to setup new node from scratch. Furthermore, I went through and created AppArmor profiles for Apache which should hopefully stop any arbitrary code execution from doing anything useful. The full details are documented on the wiki, and we invite anyone interested to audit slashcode or our infrastructure (documented fully on the wiki) and provide comments.

Closing And Stats

It has been an exhausting week to say the least, but I feel we're firmly on track. As I said in the vision statement, I intend us to change the world, one person at a time, and one step at a time, and we've finally got that first step planted in the ground. Now we just need to move forward.

As something of a tradition, I normally post stats when I finish one of these letters. Unfortunately, our new varnish config skewed slashcode's internal stat counter, so to celebrate, here's the output of varnishstat, recording hits for the last 17 hours.

0+17:41:39
Hitrate ratio:        1        1        1
Hitrate avg:     0.8705   0.8705   0.8705

      110979         0.00         1.74 client_conn - Client connections accepted
      269748         0.00         4.23 client_req - Client requests received
      225438         0.00         3.54 cache_hit - Cache hits
         713         0.00         0.01 cache_hitpass - Cache hits for pass
       33527         0.00         0.53 cache_miss - Cache misses
        4899         0.00         0.08 backend_conn - Backend conn. success
         446         0.00         0.01 backend_unhealthy - Backend conn. not attempted
           3         0.00         0.00 backend_fail - Backend conn. failures
       38944         0.00         0.61 backend_reuse - Backend conn. reuses
        2019         0.00         0.03 backend_toolate - Backend conn. was closed
       40971         0.00         0.64 backend_recycle - Backend conn. recycles
       22829         0.00         0.36 fetch_length - Fetch with Length
       17505         0.00         0.27 fetch_chunked - Fetch chunked
         200         0.00         0.00 fetch_close - Fetch wanted close
           1         0.00         0.00 fetch_failed - Fetch failed
         691         0.00         0.01 fetch_304 - Fetch no body (304)

NCommander also adds: Corrected a minor factual error. We collect IPLDs on all posts. Slash also keeps an internal hit log with IPLDs for all his for 60 hours. Admins do get their IPs logged as a way to SAN check against abuse.

 
This discussion has been archived. No new comments can be posted.
Display Options Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 1) by nadaou on Tuesday March 25 2014, @05:35AM

    by nadaou (2929) on Tuesday March 25 2014, @05:35AM (#20881)

    Check out wpkn.org for a 100% user supported model.

    There's Pacifica, but a search for recent news on them might be instructional on what not to do.

    Setting up for applying for 501c[0-9] could take 3 months. Getting it may take 3 years. Tax exemption might only come with c3, which it is unclear if this group would get. I guess a news org passes the primarily edu test though. But don't take advice from IANAL, talk to the SFLC ASAP.