The Apple News app launched with much hype on September 15. I’ll always remember that date because the app’s launch was a major determinant for me to pivot Reportory, my news customization platform, from a consumer-facing service to a license play for publishers. The Apple News app hadn’t launched when I first profiled 15 other news apps over the summer, so I wanted to closely examine the app now to see if the reality of the app lived up to its expectations. Unfortunately it did not.

One month after its launch Apple reported having more than 40 million users of its news app. That’s a bit misleading, though, since all users who upgraded to iOS 9 automatically received the app. The bigger question is around usage and satisfaction (of both users and publishers). Publishers are already voicing concerns around limited ad revenue and sparse analytics, including Time Inc. CEO Joe Ripp.

From a user perspective, I have major concerns around the content algorithm. I believe it has significant flaws in delivering relevant content, thus causing the limited revenue for publishers. Don’t get me wrong: The UX of the articles is beautiful. It’s just that the main interface choosing publishers and topics has been done many times over, so I was expecting to see an innovative content recommendation engine. But that failed on numerous occasions as noted below.

This week they announced a Top Stories tab curated by editors twice daily, which I suspect is a quick fix to the issue of the poorly personalized feed. For now it’s just a mix of Yahoo’s News Digest and Flipboard. Adding new features and redoing the user interface is not fixing their core issue. If I were working on this app I’d go back to the coding behind the story selection and continue testing it until hundreds of users got nearly perfect personalized selections.

Below is my detailed account using the app on five different occasions over several weeks. A poor match means it was neither a source nor topic match. Average implies the topic was somewhat relevant but not an exact match, and strong indicates it was click-worthy — a strong match across both source and topic. As you can see, the app failed me every single time. I will check back on its progress in the months to come, but for now I can’t see a clear differentiator from Flipboard and I’m not compelled to make it my go-to news app in the near future.

Upon launch, I selected 15 preferred channels and 16 topics. My initial sources were Bloomberg, Business Insider, CNN, Conde Nast Traveler, The Economist, New York Magazine, The New York Times, The New Yorker, NPR, Quartz, Re/Code, Vox, The Wall Street Journal, The Washington Post and Wired. I chose the following topics: Computing and Information Technology, Digital Media, Entertainment and Media Technology, Entrepreneurship, Facebook, Google, Interactive Media, Media Industry, Online Media Industry, Silicon Valley, Software Industry, Startup Company, Startups, Tech Industry, Technology and Venture Capital.

First visit: Of the first 10 articles shown to me, I would categorize nine of them as being a poor match and one a strong match. I clicked the strong match and four other relevant articles further down the page to see how that might affect the next day’s selections.

Second visit: I opened my app to find more than 100 new articles presented to me. Of the top 10, nine I graded a poor match and had no interest in reading. Most startling was the first article in the header promo spot on an air show in Dubai. This is such a poor match of my interests that I am baffled as to how it got the prime real estate. I’m not at all impressed with my selections so I clicked the one strong match,   then I go exploring to add more interests to see if I can improve this algorithm in the days to come. I suspect many users give up at this point and do not take the time to redo their selections. Not good news for the Apple News App team!

I added the following channels: The Verge, Mashable, VentureBeat, The Guardian, Reuters, Fortune and Entrepreneur. All were featured within various topical channels. I would have selected these initially had they appeared on the initial sign-up page but I suppose I didn’t scroll long enough for them to show up serendipitously.

I also added new topics: Smartphones, Social Media, Apple Inc. and Elon Musk. I thought it odd that Musk showed up as a recommendation considering I had to manually search for Tim Cook, Larry Page, Jeff Bezos, Marissa Mayer, Mark Zuckerberg and Sheryl Sandberg.

Now I have 22 preferred channels and 26 topics. I go back to my For You tab and see I already have 113 new stories in the 30 minutes since I first opened the app. I’m a big fan of consumability so this is not exciting for me to see more than 100 new updates when I hadn’t even perused the initial story selections. I get overwhelmed and close the app, hoping the next time is a better experience.

Third visit: Today’s initial article is much better than yesterday. It’s a New York Times daily briefing, which is a smart roundup article to put in the header slot. I had a few more average matches but still only one strong match. Most interesting was the 10th article, a story on Tim Cook and Apple. This piece was a deviation from my source preference but I was fine with it since Cook and Apple were two keywords I had selected. I really like this as it incites incidental information acquisition where I never would have otherwise read this article. (I don’t follow Trusted Reviews.)

Fourth visit: This was the day news broke of the Paris attacks, yet it was not until the 9th article that I finally got coverage of it. The initial story presented to me was Bloomberg’s “Ernst & Young Loses Trial Over Audits Linked to Madoff” followed by Conde Nast Traveler’s “Apples, Rum, Brandy, and … Brown Butter?” Clearly this selection could have benefitted from a human editor overwriting the algorithm to move some of the Paris coverage to the top. I still don’t trust the algorithm’s selections as it chose an Ohio State University football story (I hate football) and an article on Chrissy Teigen’s pregnant thighs (Who is this woman?) before the Paris coverage. What a massive failure!

Fifth visit: I’m perplexed. The top story is about Pluto and its moons yet I never indicated any interest in science and astronomy. The rest is mostly political coverage on Ted Cruz and Donald Trump, yet I never indicated any interest in politics (I’m a Democrat, so even if I did have an interest in following political coverage I wouldn’t follow these politicians). I close the app and am unlikely to come back anytime soon.