Goals for SimplePie 2

Over the past four years or so, SimplePie has grown from a completely unknown set of functions sitting on top of Magpie RSS to one of the world's most popular feed parsers with thousands and thousands of users all over the world. Also in that time, SimplePie has started to outgrow its architecture. People use SimplePie for all sorts of tasks that we never really anticipated, so we believe we've now reached a point where it's time for a reset.

SimplePie 2 is currently in the planning stages. The purpose of this document is to put together a list of goals for SimplePie 2 that will improve the overall project as a whole, and unshackle some of the early design decisions which now seem to be holding us back.

We encourage everyone in the SimplePie community to weigh in with their opinions on the next version of SimplePie so that we can make this software by the people, for the people. Please mark your contributions with ”~~ wiki-username” so that we know who to ask if we need clarification of the ideas. Just because it's on the list doesn't guarantee it'll get done, but we want to collaboratively share ideas and pick out the best ones.

• Modular design: We want to take a Firefox-like approach to this. The most important, most used functionality will be part of the core package, and everything else can be added a là carte (like Firefox extensions). This allows for a clearer separation of components, as well as the ability to write plugins and other custom modules. Because of this design, ALL functionality is comprised of separate modules, but a small group of modules will make up the “core package.” ~~ skyzyx
• Core Module: This will include important, re-usable stuff that is shared among the core package modules. It will also manage dependencies for new modules. ~~ skyzyx
• Configuration Module: Manages the setting/getting of configuration options. Extensible so that third-party modules can have custom settings set this way. ~~ skyzyx
• IRI Module: Understanding Internationalized Resource Identifiers (IRIs) (e.g. möbius.com). ~~ skyzyx
• HTTP Module: Handles requesting data over HTTP (with proper HTTP 1.1 support), and can understand and format the response into something more usable. Based on cURL, and supports curl_multi_exec() for parallel fetching of feeds. ~~ skyzyx
• Caching Module: Extensible caching system that manages CRUD functionality, along with an actual caching plugin (file-based). ~~ skyzyx
• Character Transcoding Module: Handles on-the-fly conversions between character encodings. Will continue to use UTF-8 internally. Uses the built-in iconv support by default, but will be enhanced by mbstring support. ~~ skyzyx
• Parsing Module: Parses conformant feeds into a standard internal data structure. Uses the same namespace-based organization as SP 1.0. ~~ skyzyx
• Core API Layer Module: Translates the internal data structure into logical API methods that third-party developers interact with. This “core” module will cover the normalization of various RSS/Atom data types, and should include all supported data types. ~~ skyzyx

• PDO-based SQL caching module, SQL/Hybrid caching module (higher performing than pure SQL caching, but more complex), APC caching module, Memcache caching module (?). Will sit on top of the core Caching module. ~~ skyzyx
• Ultra-Liberal Feed Locator. Will support a variety of methods above and beyond simple auto-discovery for discovering feeds. ~~ skyzyx
• Media RSS API layer, iTunes RSS API layer, GeoRSS API layer, hAtom API layer, Digg API layer, Y! Weather API layer, Google Calendar API layer, and others. ~~ skyzyx
• Data Sanitization module. Strips potentially dangerous content, resolves relative paths/URLs, etc. ~~ skyzyx
• Content munging modules such as text shortening, extracting images, and other related things. ~~ skyzyx
• Caching content images. Will sit on top of the core Caching module. ~~ skyzyx
• Finding and caching favicons. Will sit on top of the core Caching module. ~~ skyzyx
• JSON Web Service module that translates the internal data structure into JSON and can serve it efficiently using REST-style methods (same as SimplePie Live!) ~~ skyzyx
• Microformat module that allows for parsing HTML documents for microformats – specifically hAtom.

We've had a fairly difficult time finding new developers to help grow this project. The (relatively) recent additions of Michael Shipley and Ryan McCue have been great, but we'd like to grow the team even more to spread the development load (since this a part-time project for everyone involved). I (Ryan P.) think that part of the problem is a 10,000-line PHP file. I also think it's due to a lack of complete, in-code documentation.

We've also run into problems where people have wanted to integrate parts of SimplePie into their projects without using the whole thing. At the same time, we've heard people say that one of the best parts of SimplePie is that everything is self-contained in a single PHP file so they only need to worry about a single include.

My suggestion is that, during development, we break out individual modules and work on them independently (I'm definitely open to better ideas about this structure). Modules will need to be included one-by-one, but we could also have a “core” identifier that would load ALL core modules (which should be good for most users, most of the time). Hopefully this development approach would be less intimating to would-be contributors, so people can ease-in a little better.

• SimplePie folder
  • simplepie.inc.php (yes, with a .php file extension ;) )
  • modules (introduce a fundamentally new sort of functionality to SimplePie)
    • configuration.module.php
    • iri.module.php
    • http.module.php
    • cache.module.php
    • character.module.php
    • parse.module.php
    • api.module.php
    • json.module.php (non-core module)
    • microformat.module.php (non-core module)
  • extensions (enhancements to existing modules, or use modules as a base)
    • file.cache.module.php
    • pdo.cache.module.php
    • mediarss.api.module.php
    • hatom.microformat.module.php

When people want to download SimplePie 2, we can provide a MooTools-like approach:

• All-in-one
  • Core-only (default)
  • Everything
  • A là carte modules
• Separate files
  • Core-only
  • Everything
  • A là carte modules

• PHP 5.1.x (which includes iconv)
• PCRE (regular expression support)
• DOMDocument (better than SimpleXML at handling malformed HTML/XML markup)
• Recent version of cURL (for fetching remote data)
• Raise the knowledge requirement to something sensible. People should already be able to know how to write and call simple functions, jump in-and-out of PHP blocks, write to the page (i.e. output buffer), define simple arrays (indexed and associative), know how to utilize existing constants, and understand the basic parent-child nature of objects and methods/properties. SimplePie is a toolkit for PHP developers. It needs to start acting like one.

• Everything listed in “Bare Functionality”, plus:
• mbstring for enhanced character transcoding.
• PDO + MySQL or another supported SQL method for database caching.
• APC or Memcached for an alternative, memory-based caching method.