Technical problems and changes with 311 Web site raise questions about complaint tracking accuracy

By LOUIS FLORES

Community activists in Queens have begun to notice technical problems with the city's 311 Web site, where New York City makes available online its complaint management system.

Approximately two days ago, Peter Condiles, an activist, who lives in the Community Board 7 area of Queens, began experiencing problems with the captcha generator on the 311 Web site.  A captcha generator is a widget that ensure that users of Web sites are human and not automated programs.  Mr. Condiles informed Progress Queens that the 311 Web site had recently abandoned a captcha generator based on the Latin number and alphabet system (that we are all accustomed to) and, in its place, initiated a picture-based captcha, where 311 Web site users were asked to check off pictures, such as of street signs or food, from a panel of nine different pictures.  For Mr. Condiles, the new captcha generator would not work.  He said he has tried almost a dozen times to report a non-functioning light fixture at a bus stop and what he believed to be illegal construction, yet his attempts to report these complaints using the 311 Web site have failed as a result of the new, image-based captcha generator.  When asked by Progress Queens if Mr. Condiles was using any non-traditional means of accessing the 311 Web site, Mr. Condiles replied that he was completing the 311 online form using his mobile phone, which had worked in the past.

"I have never had this problem with the 311 system with this phone," Mr. Condiles told Progress Queens, adding that, "I've had this phone for five years now."  

Mr. Condiles also told Progress Queens that he had noticed that the complaint count on the 311 Web site's live map of the five boroughs appeared to be about half of what the numbers used to be reported.  

Another prominent Queens community activist, Joe Moretti, publisher of the Cleanup Jamaica Queens Now Web site, has also experienced his own technical problems following the recent changes to the 311 Web site.   

Amongst the problems that Mr. Moretti told Progress Queens that he has encountered include :  fewer categories under which complaints could be made ; the default entry of date and time that the Web site enters for complaints are sometimes in the future ; the rejection of existing streets or intersections ; and the appearances of error messages at the conclusion of having entered all of the complaint information, rendering his entire complaint "null," he said.

"They've made it difficult to use.  It's not as user-friendly as before," Mr. Moretti said, adding in respect of the changes in categories, "What I want to complaint about, I cannot find it."

When the 311 Web site gave him a future date and time, Mr. Moretti said that he would have to manually change the date and time, or else he would have to back out of the page and then return, in order to re-enter his information, saying, "That happens nine out of ten times."  Regarding the 311 Web site's rejection of known street names or intersections (for the location of his complaints), Mr. Moretti said to Progress Queens, "The system will tell me no such intersection or street exists.  What's the deal with this ?"  Mr. Moretti said that he noticed these problems ever since the 311 Web site was revamped.

A request for an interview made by Progress Queens to the City Hall press office about the 311 system was not answered by Natalie Grybauskas, a City Hall press officer. 

Another prominent Queens community activist, who advocates in respect of civic issues on behalf of Queens residents as the anonymous operator of the Queens Crap Web site, informed Progress Queens that there have also been problems with 311's smart phone-based application.  The 311 app will attempt to add a smart phone's physical location using the smart phone's GPS information into a complaint being made on the 311 app.  However, there are times when the 311 app user will not be at the physical location, which is the subject of the complaint.  At times, attempts to correct the physical location will result in failure.  At times, the 311 app will generate error messages, such as the loss of an Internet signal, creating a frustrating experience for 311 app users, leading some users to abandon attempts to report complaints, the Queens Crap Web site administrator informed Progress Queens.

Progress Queens tests the 311 Web site

The 311 Web site was unable to process the first attempt made by a reporter for Progress Queens to report an incident of dog waste left on a sidewalk.  Source :  311 Web site/NYC.gov 

The 311 Web site was unable to process the first attempt made by a reporter for Progress Queens to report an incident of dog waste left on a sidewalk.  Source :  311 Web site/NYC.gov 

After Progress Queens began to receive information about technical problems with the 311 Web site, Progress Queens conducted an independent test of the 311 Web site.  

Uncollected dog waste on the sidewalk outside The Berkeley apartment house in Jackson Heights, Queens.  Source :  Louis Flores/Progress Queens

Uncollected dog waste on the sidewalk outside The Berkeley apartment house in Jackson Heights, Queens.  Source :  Louis Flores/Progress Queens

Since the only way to test the 311 Web site was to report a complaint, a reporter from Progress Queens walked around the neighborhood of Jackson Heights, Queens, to see if there were any real incidents to report.  Given that Jackson Heights is a quiet community, the only incidents that could be reported after a 30-minute walk around the neighborhood were two situations, where dog walkers had left dog waste on sidewalks near two large apartment houses :  in front of The Berkeley, located at 35-24 78th Street, and near the 79th Street service entrance to Jefferson Hall, located at 78-12 35th Avenue.

Using a computer-based browser, the first attempt to report a complaint about The Berkeley failed without triggering a need to complete the captcha generator.  A screenshot of the error message appears above.  Using the same computer-based browser, a second attempt to report the complaint about The Berkeley was processed after the captcha generator did not trigger any challenge to complete, producing a confirmation number.  Using a different computer-based browser, an attempt to report a complaint about Jefferson Hall was processed after the captcha generator triggered not one, but two picture-based challenges (one for street signs and another for oranges) that were nonetheless successfully completed, producing a confirmation number.

There is no explanation for why the first computer-based browser would end in two different outcomes or for why the second computer-based browser would trigger a third experience. 

The live map of complaints

The live map on the 311 Web site that tracks reporting of 311 complaints on a rolling basis only shows recently open and recently closed complaints.  According this methodology, the number of complaints appears to be being manipulated to construct an artificially low number of complaints.  A review conducted on Friday morning by Progress Queens revealed that, according to this narrow calculation methodology, the peak number of complaints for any community district was 777, in this instance for the Upper East Side.

The live map of the 311 Web site's tally of recently opened and recently closed complaints calculated the peak number of complaints in any one community district at 777, for the Upper East Side of Manhattan.  Source :  311 Web site/NYC.gov

The live map of the 311 Web site's tally of recently opened and recently closed complaints calculated the peak number of complaints in any one community district at 777, for the Upper East Side of Manhattan.  Source :  311 Web site/NYC.gov

The criteria for the determination of the live map calculation is based on complaints that are open and have been recently closed (within the last five days).  Five days allows complaints that may be closed on the same day as they are filed to age out of the map's calculations within the same week as they are filed.

Progress Queens conducted monthly searches of the 311 Web site's complaint reporting.  The 311 Web site's search function does not allow monthly comparisons to the previous year.  For the following months, the 311 Web site reported much higher peak reporting numbers than the 777 complaints on the live map :

  • 2015 October :  2,840
  • 2015 September : 3,579
  • 2015 August : 3,561
  • 2015 July :  3,609
  • 2015 June :  3,759
  • 2015 May :  3,674
  • 2015 April :  2,958
  • 2015 March :  3,091
  • 2015 February :  2,610
  • 2015 January :  2,574
  • 2014 December :  2,536

The 777 complaints being represented on the live map constitute between one-fourth to one-fifth of the total number of complaints in some of the recent months, or approximately a week's worth of complaints, in some instances.

There was no way to independently verify the calculation methodology of the former version of the live map, which the activist Mr. Condiles said used to report approximately double the number of complaints than the current live map.

It's also not known the total scope of changes made to the 311 Web site, given City Hall's pattern of a media blackout directed at Progress Queens.

The 311 app has also become the subject of complaints, particularly about technical difficulties related to automatic GPS locationing and error messages, such as loss of Internet signal.  Not all reporting functions are operational on the 311 app, such as reporting traffic lights that are in disrepair, which must still be prorated by telephone call.  Source :  The 311 App/NYC.gov

The 311 app has also become the subject of complaints, particularly about technical difficulties related to automatic GPS locationing and error messages, such as loss of Internet signal.  Not all reporting functions are operational on the 311 app, such as reporting traffic lights that are in disrepair, which must still be prorated by telephone call.  Source :  The 311 App/NYC.gov

Last August, the journalist James Fanelli filed a report for DNAinfo showing that 311 complaints about homelessness had risen approximately 60 per cent. since Mayor de Blasio took office.  That DNAinfo report triggered an onslaught of negative media coverage for the de Blasio administration, as reflected in a report published by The New York Daily News and a report published by The New York Post.  

(The resulting avalanche of bad press about quality of life issues have placed the de Blasio administration on the political defensive, culminating this week in a show-down between the media and the de Blasio administration when WCBS-Channel 2 news reporter Marcia Kramer confronted Mayor de Blasio and, later, his chief press aide, Karen Hinton, over unanswered questions about homelessness.)

As a result of the de Blasio administration's political sensitivity to news reports about increasing problems with quality of life issues, some Queens activists believed that the changes to the 311 Web site, and the resulting technical difficulties, either accidental or intentional, could be having a desired political result by artificially decreasing the number of complaints being filed with the city.

Of motivations by the de Blasio administration to deliberately make changes to the 311 Web site to decrease reporting of complaints, Mr. Condiles said, "That's a possibility.  I couldn't prove it, but that's possible."

Regarding the possible political motivations that may make the number of 311 complaint reports a sensitive subject for Mayor de Blasio and the issues with technical reliability of the 311 Web site, Mr. Moretti said, "I believe this system is rigged somehow."  Whether the technical difficulties with the 311 Web site were accidental or intentional, Mr. Moretti noted that the outcome was the same in respect of public officials taking no meaningful action to deal with some of the city's quality of life issues, saying, "Nothing ever gets done about it," rhetorically asking later in the interview that, "Why isn't anything being done to stop the problems from happening again ?"

Mr. Moretti said that New Yorkers don't use the 311 complaint system, because they want to.  They use it, because they have to.  "We don't want to file complaints all the time," Mr. Moretti said, addressing his concerns to public officials, adding, "Why do we file complaints ?  Because you're not addressing the problems in the proper way."

311 transfers some complaints to other agencies

Further tests by Progress Queens of the 311 Web site revealed that some complaints are not even issued 311 confirmation numbers.  Since only real 311 complaints can be made online, a reporter for Progress Queens set out to report two real complaints.

First, a reporter tried to make an online complaint about the noise of airplane jets that depart from LaGuardia Airport.  Under certain departure routes, some jets make their initial turn after lift-off directly over 77th Street in Jackson Heights, Queens.  On Friday morning, after a jet made a loud turn directly overhead of a reporter's apartment, the reporter attempted to submit a 311 complaint by completing the online form, but before the reporter could enter meaningful information about the complaint, the reporter was forced to exit the 311 Web site and was directed, instead, to the Web site of the Port Authority, the operator of the city's airports, at which point the Progress Queens reporter ceased his attempt to make the online complaint report.

The Progress Queens reporter then returned to the 311 home page and attempted to make a complaint about the reporter's Metropolitan Transportation Authority MetroCard, which expired last week with a small remaining balance.  In the time since, the Progress Queens reporter made two unsuccessful attempts with two different MTA ticket booth agents to transfer the balance on the expired MetroCard to a new MetroCard.  When the Progress Queens reporter began to fill out the 311 complaint form, the reporter was forced to exit the 311 Web site and was directed, instead, to the MTA Web site, at which point the Progress Queens reporter ceased his attempt to make the online complaint report.

Reports about the quality of airport travel and use of the subway system are often reported about in the mainstream media, particularly during nightmare scenarios when jets remain on airport taxiways for hours before departure, or when conditions lead to loss of service or severe delays in the subway system.  None of these complaints would ostensibly be captured, reported, or reviewed by the 311 online complaint management system.

When Mr. Moretti was asked by Progress Queens for his opinion about the design of the 311 Web site that transfers complaints to other agencies, Mr. Moretti said, "That's a very shady thing, because that changes the numbers of the 311 system.  It passes the buck," adding that, "When you look at the overall numbers, they are not going to count all of the complaints."

In summary, Mr. Moretti said that the 311 system was design to track the people's complaints, adding that changes to the 311 system are subverting the system's original intention, saying, "That's the purpose of 311 :  That system should take into account all of the complaints."

Reference Documents