Stories
Slash Boxes
Comments

Dev.SN ♥ developers

Testing Factors for Character Entity and UTF-8 fixes

posted by martyb on Thursday November 20 2014, @01:39PM   Printer-friendly
from the what-goes-where dept.
Code

martyb writes:

Test factors for validating implementation of character entities and UTF-8 characters throughout SN.

These are the variables to permute:

  1. Characters under test: (cf: Trying some UTF-8 characters; POT, 0x0000 - 0x07ff POT, and 0x0800 - 0x0fff; POT)
    1. ampersand,
    2. less-than,
    3. greater-than,
    4. quote,
    5. Sampling of other named character entities,
    6. sampling of UTF-8 1-octet chars,
    7. sampling of UTF-8 2-octet chars,
    8. sampling of UTF-8 3-octet chars,
    9. sampling of UTF-8 4-octet chars (requires MySQL database update).
  2. Ordering of less-than and greater-than characters. (Slashcode has processing to strip HTML end elements which lack matching start elements.)
  3. Character representation:
    1. literal character,
    2. named character entity,
    3. decimal numeric character entity,
    4. hexadecimal numeric character entity.
  4. Text entry location (places where user input is obtained for later display):
    1. Story Submission (e.g. http://soylentnews.org/submit.pl):
      1. "Your Name",
      2. "Your Email or Homepage",
      3. "Title",
      4. "The Scoop",
    2. Editor's story processing (e.g. http://soylentnews.org/submit.pl?op=viewsub&subid=4848&title=Coal+Plants+Get+New+Lease+on+Life+with+Natural+Gas and then after Preview: (http://soylentnews.org/admin.pl) and then after approval, go in and edit the story again: (http://soylentnews.org/admin.pl?op=edit&sid=14/11/20/1556209.)
      1. "Dept" (e.g. from the hyphenated-words dept.),
      2. "Intro Copy" text area,
      3. "Extended Copy" text area,
      4. "Media" text area,
    3. Comment reply to a story (http://soylentnews.org/comments.pl?sid=4950&op=reply&threshold=-1&commentsort=0&mode=improvedthreaded#post_comment:
      1. "Subject",
      2. "Comment" text area
    4. Journal article submission: (http://soylentnews.org/my/journal and then click Write in Journal):
      1. "Subject",
      2. "Entry" text area,
    5. Journal Reply/Comment entry
      1. "Subject",
      2. "Comment"
    6. Poll creation (cf: http://soylentnews.org/pollBooth.pl) (Need help here; never submitted a poll before!)
      1. text(?),
      2. poll comment(?)
    7. Poll responses:
      1. "Subject" (??),
      2. "Comment" (??),
      3. Other (??)
    8. User preferences (http://soylentnews.org/my/info — "User" tab:
      1. "Real Name",
      2. "Email Address",
      3. "Homepage",
      4. "AIM",
      5. "ICQ UIN",
      6. "Yahoo! ID",
      7. "Jabber",
      8. "Public Calendar",
      9. "Mobile Text Address",
      10. "Sig" (120 chars),
      11. "Bio" (255 chars).
    9. Homepage - "User Space" text area (http://soylentnews.org/my/homepage) i.e. "If you choose 'User Space' above, the contents of this box will be inside:"
    10. Anything else?
  5. Story submission / comment submission format:
    1. "Plain Old Text",
    2. "HTML Formatted",
    3. "Extrans (html tags to text)",
    4. "code"
  6. Text viewing location:
    1. On-line web page,
    2. Print formatted story (printer icon) (e.g. http://soylentnews.org/print.pl?sid=14/11/20/1556209),
    3. e-mailed story,
    4. web message,
    5. RSS feed,
    6. Atom feed,
    7. "Authors page" (e.g. http://soylentnews.org/authors.pl),
    8. Author's Story list page (e.g. http://soylentnews.org/search.pl?op=stories&author=76),
    9. News Feed (NNTP) (How do we do this?),
    10. "Palm" (How do we do this?),
    11. Search results page (e.g. from using: http://soylentnews.org/search.pl).
    12. User's Info Page: (e.g. http://soylentnews.org/~martyb/),
      1. Comment list section as displayed here,
      2. Additional comments: next batch of comments and following,
      3. Recent Submissions section.
      4. PollBooth slashbox
      5. Poll Results (e.g. http://soylentnews.org/pollBooth.pl?qid=51&aid=-1
  7. Preview count: how many times a user has previewed a story submission or [journal|story] comment prior to submitting (preview has had some issues in the past.):
    1. Once,
    2. Twice,
    3. Thrice

If I have missed any user-visible input/output/modification location, please reply and let me know!

Related Stories

by
by
by
SLASHLINK bug - test story 6 comments

martyb writes:

An AC reported that they saw (parameterized here; replace text with corresponding character and remove all spaces):

underscore underscore SLASHLINK underscore underscore

when they viewed all comments on a story and/or tried to reply to a comment.

From an investigation of the code, it appears to be triggered by the setting of the slash var "article_link_story_dynamic" — on dev it's set to "1" but I cannot see its setting on production.

This post is an attempt to reproduce the problem.

The problem arises when a story includes a reference to another story on the site (if I understand correctly: the url contains a reference to article.pl)

So, here's a link to one: http://dev.soylentnews.org/article.pl?sid=14/11/20/1737231 (which linked to the story: "Testing Factors for Character Entity and UTF-8 fixes")

A quick review of (an old copy of) the code suggested the site var "article_link_story_dynamic" may be the culprit:

plugins/Relocate/Relocate.pm:                                next if $token->[1]{href} eq '__SLASHLINK__';
Slash/Utility/Data/Data.pm:                Slash/Utility/Data/Data.pm:                $retval = q{  
Slash/Utility/Data/Data.pm:HTML with slashized links (see slashizedLinks()) and converts them to
 
Slash/Utility/Data/Data.pm:                _slashlink_to_link($1, $options)
Slash/Utility/Data/Data.pm:sub _slashlink_to_link {
Slash/Utility/Data/Data.pm:# behavior of _slashlink_to_link.
 
Slash/Utility/Data/Data.pm:                # Set article_link_story_dynamic to 2 or greater and
Slash/Utility/Data/Data.pm:                my $force_dyn = $constants->{article_link_story_dynamic} > 1 ? 1 : 0;
Slash/Utility/Display/Display.pm:    my $dynamic = $constants->{article_link_story_dynamic} || $story_link->{dynamic} || 0;
sql/mysql/upgrades:6143:INSERT INTO vars (name, value, description) VALUES ("article_link_story_dynamic", "0", "Change default dynamic status for story linking");

This discussion has been archived. No new comments can be posted.
Testing Factors for Character Entity and UTF-8 fixes | Log In/Create an Account | Top | 1000 moderator points   | 2 comments | Search Discussion
Display Options Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

  • (Score: 2) by The Mighty Buzzard on Thursday November 20 2014, @02:18PM

    by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@soylentnews.org> on Thursday November 20 2014, @02:18PM (#28198) Journal
    We're again not looking at input but output. Filters have been flat removed from input everywhere I could find them. What we're missing is everywhere user-entered data can be displayed. I did miss polls though. Email messages are working as expected, web messages are not. Having issues finding where messages are created though, so that's staying broken until I do.
    --
    123
    456
    789

    • (Score: 2) by martyb on Monday November 24 2014, @11:50PM

      by martyb (76) on Monday November 24 2014, @11:50PM (#28203) Journal

      First off, I apologize for taking so long to reply to your comment. I read your comment but lacked time to reply at that moment, and failed to give it a priority to reply until now.

      Separately, we're coming up on Black Friday which translates in the retail world to mean "you can pretty much forget about any normal kind of hours for a while." So, my availability (and likely my ability, too) will be rather curtailed until at least a couple weeks into the new year. I don't intend to fall off the face of the earth, so to speak, but my hours here are likely to be severely curtailed. :(

      We're again not looking at input but output.

      Yes, point well-made and well-taken!

      Filters have been flat removed from input everywhere I could find them. What we're missing is everywhere user-entered data can be displayed.

      I misunderstood that point, before. There is coverage of that in point "6. Text viewing location:", above. Off-hand, I can think of no other places. That said, I'd like to make another pass through the site to see if I can find any other places where user-entered input is output.

      On the other hand, let's assume we *have* found all the places that user-entered data can be displayed. How do we test it? We'll need to know where to enter user data to get it to come out there, right? So, I'm holding onto the inputs I documented.

      PS. As a thought experiment, what if we started with a minimalist install of the site in a VM, entered unique and distinct values in each of these input fields, and then looked to see where each of these came out? One could use a combination of a web-site scraper, e-mail client, NNTP client, etc. to view all the possible outputs. Though I'm not advocating actually doing so for all possible inputs, it may prove to be viable for a selection of inputs/outputs when it comes to coverage testing and the like.