2.3M

URLs processed

Pipfeed's parse API can process any type of article, blog, text.

100%

Service Uptime

An API you can rely on in your production app. We never go down.

9.1/10

Popularity on RapidAPI

Our customers love us, we are one of the premier Article & content parsing APIs on RapidAPI.

500ms

API Response Time

We parse any URL at the speed of the page load time. Our parsing engines are optimized to return results fast.

Try out Pipfeed's extract API

Enter any news article URL to parse.

00
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
{
  "url": "https://medium.com/pipfeed/how-does-pipfeeds-recommendation-engine-work-pipfeed-com-2f3c12546fb",
  "title": "How does Pipfeed’s recommendation Engine work? — PipFeed.com",
  "author": "Shashank Agarwal",
  "length": 1420,
  "siteName": "PipFeed",
  "description": "At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create…",
  "html": "<div class=\"page\" id=\"readability-page-1\"><div><figure><div tabindex=\"0\" role=\"button\"><p><img role=\"presentation\" src=\"https://miro.medium.com/max/1400/0*ySiYrjZSIUZC_fyT.jpg\" alt=\"\"></p></div></figure><p id=\"5ee3\">At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create a recommendation engine and even hard to measure how well the algorithm is working. Here is how Pipfeed’s recommendation engine works.</p><p id=\"b56b\">The main feed is comprised of “Plugins”. Each of these plugins “competes” to add articles to the user’s feed.</p><h2 id=\"6954\">Pipfeed’s plugins</h2><p id=\"6937\">Pipfeed has a plugin system internally and each of the plugins have different approach to add articles to the user’s main feed.</p><ul><li id=\"bde2\"><strong>Subscriptions:</strong> Add articles from the blogs user has subscribed to</li><li id=\"1e39\"><strong>Interests:</strong> Adds articles from user’s Interests on their profile</li><li id=\"406a\"><strong>Trending:</strong> Adds articles that are trending on PipFeed</li><li id=\"3ed2\"><strong>Past Liked Blogs</strong>: Adds articles from user’s previous most liked blogs</li><li id=\"a6e6\"><strong>Past Liked Interests:</strong> Adds articles from user’s past liked Interests</li><li id=\"268c\"><strong>Collaborative filtering:</strong> Adds articles similar to previously read articles</li><li id=\"ab18\"><strong>Content-based filtering:</strong> Find users with similar reading behavior and then adds from that user’s past history</li><li id=\"2fe3\">And more…</li></ul><h2 id=\"568c\">The Feedback Loop</h2><p id=\"5b8c\">Pipfeed’s recommendation engine re-balances the importance given to each plugin based on user interactions. So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed.</p><p id=\"824c\">So the system keeps “rebalancing” itself over time and as the user uses the app more, the more personalized the feed becomes.</p></div></div>",
  "text": "At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create a recommendation engine and even hard to measure how well the algorithm is working. Here is how Pipfeed’s recommendation engine works.The main feed is comprised of “Plugins”. Each of these plugins “competes” to add articles to the user’s feed.Pipfeed’s pluginsPipfeed has a plugin system internally and each of the plugins have different approach to add articles to the user’s main feed.Subscriptions: Add articles from the blogs user has subscribed toInterests: Adds articles from user’s Interests on their profileTrending: Adds articles that are trending on PipFeedPast Liked Blogs: Adds articles from user’s previous most liked blogsPast Liked Interests: Adds articles from user’s past liked InterestsCollaborative filtering: Adds articles similar to previously read articlesContent-based filtering: Find users with similar reading behavior and then adds from that user’s past historyAnd more…The Feedback LoopPipfeed’s recommendation engine re-balances the importance given to each plugin based on user interactions. So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed.So the system keeps “rebalancing” itself over time and as the user uses the app more, the more personalized the feed becomes.",
  "topImage": "https://miro.medium.com/max/1024/0*ySiYrjZSIUZC_fyT.jpg",
  "date": "2020-11-11T05:22:00.807Z",
  "keywords": [
    "recommendation engine",
    "Artificial Intelligence",
    "Engineering",
    "Computers and the Internet",
    "Tech Industry"
  ],
  "summary": [
    "Here is how Pipfeed’s recommendation engine works",
    "The main feed is comprised of “Plugins”.",
    "Each of these plugins “competes” to add articles to the user’s feed",
    "So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed"
  ]
}

Parse any URL

This includes news articles, blogs, JS blogs, papers, and more ...

Parse article behind Ad-Walls

Parse New York Times, Techcrunch, and more.
https://www.nytimes.com/2022/08/26/technology/crypto-ethereum-the-merge.html

00
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
{
  "url": "https://www.nytimes.com/2022/08/26/technology/crypto-ethereum-the-merge.html",
  "title": "The Crypto World Can’t Wait for ‘the Merge’",
  "author": "David Yaffe-Bellany",
  "length": 9677,
  "siteName": "New york Times",
  "description": "A long-awaited upgrade to Ethereum, the most popular crypto platform, may make the technology more environmentally sustainable. But it comes with risks.",
  "html": "<div class=\"page\" id=\"readability-page-1\"><article id=\"story\"><div id=\"fullBleedHeaderContent\"><header><div><div><p><img alt=\"Cinemagraph\" src=\"https://static01.nyt.com/images/2022/08/23/autossell/still_01_Fin/still_01_Fin-square640.jpg\"></p><video playsinline=\"\" autoplay=\"\" loop=\"\" muted=\"\" src=\"https://vp.nyt.com/video/2022/08/23/102072_1_00merge_wg_480p.mp4\" data-testid=\"cinemagraph\"></video></div><figcaption><span><span>Credit...</span><span>Timo Lenzen</span></span></figcaption></div><div><p>A long-awaited upgrade to Ethereum, the most popular crypto platform, may make the technology more environmentally sustainable. But it comes with risks.</p></div></header><p aria-hidden=\"true\"><span></span><span><span>Credit...</span><span><span>Timo Lenzen</span></span></span></p><div><div><p><span><a href=\"http://www.nytimes.com/by/david-yaffe-bellany\">David Yaffe-Bellany</a></span></p><p>David Yaffe-Bellany, who covers crypto, reported this article from San Francisco.</p></div><ul><li><time datetime=\"2022-08-26T05:00:22-04:00\">Aug. 26, 2022</time></li></ul></div></div><section name=\"articleBody\"><div><p><a title=\"\" href=\"https://www.nytimes.com/interactive/2022/03/18/technology/cryptocurrency-crypto-guide.html\">The cryptocurrency industry</a> has endured a terrible year. A devastating crash wiped nearly $1 trillion from the market, <a title=\"\" href=\"https://www.nytimes.com/2022/06/29/technology/crypto-crash-divide.html\">draining the savings</a> of thousands of people. Several companies <a title=\"\" href=\"https://www.nytimes.com/2022/08/18/technology/crypto-crash-investments.html\">filed for bankruptcy</a>.</p><p>Now the industry is fixated on a potential saving grace: a long-awaited software upgrade to the most popular cryptocurrency platform, <a title=\"\" href=\"https://www.nytimes.com/2017/10/01/technology/what-is-ethereum.html\">Ethereum</a>, which provides the technological backbone for <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.bloomberg.com/news/articles/2022-08-24/crypto-confronts-own-y2k-moment-with-ethereum-network-upgrade?sref=zVYYYI5e\">thousands of crypto projects</a>. The upgrade — known as the Merge — has gained near-mythical status after years of delays that left some insiders questioning whether it would ever happen.</p><p>But if all goes according to plan, the Merge will take place around Sept. 15, more than eight years after it was initially discussed. The change would shift Ethereum to a more energy-efficient infrastructure, addressing the <a title=\"\" href=\"https://www.nytimes.com/2022/03/22/technology/bitcoin-miners-environment-crypto.html\">widespread criticism</a> that crypto’s climate impact outweighs its possible benefits. And it would lay the foundation for future upgrades to reduce the hefty fees required to conduct transactions in Ether, the platform’s signature currency and the second-most valuable digital asset after Bitcoin.</p></div><div><p>“This transition is essentially laying a road map to a future which is far more scalable, far more energy efficient and a lot more usable to the normal person,” said Joseph Ayoub, an analyst at Citi who has studied the Merge. “It’s laying the groundwork for adoption.”</p></div><div><p>But the risks are profound. Even by crypto standards, the process is almost ludicrously complicated.</p><p>For months, insiders have engaged in frantic, jargon-heavy discussions of developments such as the <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.coindesk.com/tech/2022/08/11/ethereums-third-and-final-testnet-merge-goes-live-on-goerli/\">Goerli Testnet Merge</a> and the <a rel=\"noopener noreferrer\" title=\"\" href=\"https://news.yahoo.com/ethereum-finalizes-start-date-bellatrix-083639621.html\">Beacon Chain’s Bellatrix upgrade</a>, crucial software changes leading up to the main event. A botched Merge would imperil the thousands of crypto applications that operate using Ethereum, which collectively handle more than <a rel=\"noopener noreferrer\" title=\"\" href=\"https://dappradar.com/defi/protocol/ethereum\">$50 billion</a> in user funds.</p><p>“It’s flying the jet, and changing the engine in the sky,” said Chandler Guo, a crypto industry veteran who <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.bloomberg.com/news/articles/2022-08-06/crypto-miner-pushback-suggests-ethereum-offshoots-are-coming?sref=zVYYYI5e\">leads a group opposing the Merge</a>. “It’s very difficult. It’s very dangerous.”</p></div><div data-testid=\"photoviewer-wrapper\"><figure role=\"group\" aria-label=\"media\"><div><p><span>Image</span></p><picture><source srcset=\"https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-mobileMasterAt3x.jpg?quality=75&amp;auto=webp&amp;disable=upscale&amp;width=600\" media=\"(max-width: 599px) and (min-device-pixel-ratio: 3),(max-width: 599px) and (-webkit-min-device-pixel-ratio: 3),(max-width: 599px) and (min-resolution: 3dppx),(max-width: 599px) and (min-resolution: 288dpi)\"><source srcset=\"https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-mobileMasterAt3x.jpg?quality=75&amp;auto=webp&amp;disable=upscale&amp;width=1200\" media=\"(max-width: 599px) and (min-device-pixel-ratio: 2),(max-width: 599px) and (-webkit-min-device-pixel-ratio: 2),(max-width: 599px) and (min-resolution: 2dppx),(max-width: 599px) and (min-resolution: 192dpi)\"><source srcset=\"https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-mobileMasterAt3x.jpg?quality=75&amp;auto=webp&amp;disable=upscale&amp;width=1800\" media=\"(max-width: 599px) and (min-device-pixel-ratio: 1),(max-width: 599px) and (-webkit-min-device-pixel-ratio: 1),(max-width: 599px) and (min-resolution: 1dppx),(max-width: 599px) and (min-resolution: 96dpi)\"><img height=\"400\" width=\"600\" decoding=\"async\" sizes=\"((min-width: 600px) and (max-width: 1004px)) 84vw, (min-width: 1005px) 80vw, 100vw\" srcset=\"https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-articleLarge.jpg?quality=75&amp;auto=webp 600w,https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-jumbo.jpg?quality=75&amp;auto=webp 1024w,https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-superJumbo.jpg?quality=75&amp;auto=webp 2048w\" src=\"https://static01.nyt.com/images/2022/08/27/business/00merge-1/00merge-1-articleLarge.jpg?quality=75&amp;auto=webp&amp;disable=upscale\" alt=\"Banks of computers mining cryptocurrencies in Argentina. The Merge would shift Ethereum to a more energy-efficient infrastructure.\"></picture></div><figcaption><span><span>Credit...</span><span><span aria-hidden=\"false\">Sarah Pabst for The New York Times</span></span></span></figcaption></figure></div><div><p>Ethereum is a <a title=\"\" href=\"https://www.nytimes.com/2018/06/27/business/dealbook/blockchains-guide-information.html\">blockchain</a>, a publicly viewable ledger where exchanges of digital coins are recorded. Transactions on the chain are conducted in Ether.</p></div><div><p>The platform was started in 2013 by a teenage programmer, Vitalik Buterin, who is now regarded as one of the industry’s <a rel=\"noopener noreferrer\" title=\"\" href=\"https://time.com/6158182/vitalik-buterin-ethereum-profile/\">elder statesmen</a>. Mr. Buterin wanted to create a crypto system that was more <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.wired.com/2014/01/ethereum/\">flexible</a> than Bitcoin and could instantly execute financial contracts and other complex forms of exchange.</p><p>Ethereum’s design allows it to support a range of financial engineering. Programmers can build applications using the software to perform more complicated tasks than simple money transfers. Thousands of businesses and projects in the experimental world of decentralized finance now use the platform to offer lending, borrowing and other sophisticated investment options. Many nonfungible tokens — the unique digital collectibles known as NFTs — are built on Ethereum.</p><p>At its core, the Merge is a change to Ethereum’s verification system. When someone sends money in a traditional transaction, a bank serves as the middleman, verifying that one person has sufficient funds to pay someone else.</p><p>Crypto operates without that middleman. In this alternate financial system, transactions are verified by a scattered network of computers. Anyone can plug a machine into the network by running software that solves complex puzzles, an energy-guzzling process for confirming transactions. Essentially, the computers are racing one another: When the puzzle is solved, the winning participants are rewarded with new coins in the digital currency they are verifying.</p><p>This verification process is widely known as crypto mining and has the technical name “proof of work.” By some estimates, the amount of energy consumed each year in mining is <a rel=\"noopener noreferrer\" title=\"\" href=\"https://ethereum.org/en/energy-consumption/#:~:text=Digiconomist%20provides%20whole%2Dnetwork%20energy,(53%20MT%2Fyr).\">comparable</a> to the annual emissions of entire countries.</p><p>The Merge is set to shift Ethereum to an alternate framework called “proof of stake,” which requires less energy. In a proof-of-stake system, computers don’t burn energy racing to verify transactions. Instead, crypto investors deposit a certain number of digital coins in a shared pool, which enters them into a lottery. Each time an exchange happens, a participant is selected from the lottery to verify the transaction and win the rewards.</p></div><div><p>The shift is expected to cut Ethereum’s energy use by more than 99 percent, which crypto boosters hope will make the technology more popular.</p></div><div><p>“The difference in hardware and energy consumption is so significant,” said Preston Van Loon, a developer working on the Merge. “When NFTs were blowing up, people were saying, ‘I’d love to have an NFT, but it feels like I’m burning a forest down.’”</p></div><div data-testid=\"photoviewer-wrapper\"><figure role=\"group\" aria-label=\"media\"><div><p><span>Image</span></p></div><figcaption><span><span>Credit...</span><span><span aria-hidden=\"false\">Stacy Kranitz for The New York Times</span></span></span></figcaption></figure></div><div><p>Switching to proof of stake could also help solve another of Ethereum’s biggest problems: the substantial fees required to use the network. Ethereum can handle only a certain amount of activity at a time, so when there’s demand for the platform, the price of using it goes up. Anyone sending Ether must pay a “gas fee” — a transaction charge that at times has risen as high as $200.</p><p>The Merge won’t eliminate that problem immediately, but developers say it <a rel=\"noopener noreferrer\" title=\"\" href=\"https://ethereum.org/en/upgrades/merge/#misconceptions\">will lay critical groundwork</a> for future upgrades designed to minimize fees.</p><p>But a change on this scale could also make Ethereum vulnerable to hacks or other disruptions, some crypto experts say. “Any time you’re making changes to a complex system, there must necessarily arise unintended consequences,” said Christopher Calicott, a crypto venture investor.</p><p>Much of the criticism is fueled at least partly by self-interest. Many of the Merge’s most vocal opponents are businesses that have <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.coindesk.com/business/2021/08/12/ethereum-miners-make-multimillion-dollar-bet-on-upgrade-delay/\">built expensive data centers</a> to mine Ether in the proof-of-work system.</p></div><div><p>The Merge’s origins date to the time of Ethereum’s creation. Mr. Buterin raised the possibility of <a rel=\"noopener noreferrer\" title=\"\" href=\"https://blog.ethereum.org/2014/07/22/launching-the-ether-sale/\">switching</a> to proof of stake in 2014, but at that point, the system was untested. The most successful cryptocurrency was Bitcoin, which uses proof of work.</p></div><div data-testid=\"photoviewer-wrapper\"><figure role=\"group\" aria-label=\"media\"><div><p><span>Image</span></p></div><figcaption><span><span>Credit...</span><span><span aria-hidden=\"false\">Michael Ciaglo/Getty Images</span></span></span></figcaption></figure></div><div><p>Since then, many newer cryptocurrencies have used proof of stake successfully. Ethereum programmers have been working in earnest on the switch for at least four years. The work is complex, and progress has been slow. Engineers had to construct a new blockchain, and run tests to check for security holes or other technological bugs that might disrupt the transition.</p><p>A nonprofit called the Ethereum Foundation helps supervise the platform. But in reality, Ethereum is run by a loose group of engineers across the world; no top-down authority orchestrated the Merge. Periodically, programmers get together on publicly streamed video calls to debate technical aspects of the shift.</p><p>At one point, the shift to proof of stake was supposed to happen as early as 2016. As the Merge came together this year, crypto enthusiasts expected a June launch date. Then the Merge was pushed back to August. Now it’s set for next month.</p><p>In the crypto world, the delays became <a rel=\"noopener noreferrer\" title=\"\" href=\"https://twitter.com/UOSking/status/1514243020874526727\">a kind of</a> <a rel=\"noopener noreferrer\" title=\"\" href=\"https://twitter.com/search?q=ethereum%20merge%20delay%20meme&amp;src=typed_query&amp;f=top\">joke</a>. Early on, engineers installed what’s known as a “difficulty bomb” in Ethereum’s code. It was designed to keep them honest: After a set amount of time, the bomb would go off, causing disruptions in the Ethereum network unless it was switched to proof of stake. But every time the bomb was supposed to go off, the engineers created a new bit of code to <a rel=\"noopener noreferrer\" title=\"\" href=\"https://news.coincu.com/98140-ethereum-breaks-promise-the-merge-delayed/\">defuse</a> it, somewhat defeating the point.</p><p>“We’re not doing this lightly,” said Danny Ryan, an Ethereum Foundation researcher who has worked on the platform since 2017. “There’s been countless amount of engineering and testing and academic vetting.”</p></div><div><p>In December 2020, Ethereum programmers took a big step toward the Merge by releasing a crypto platform called the Beacon Chain, a proof-of-stake system designed to provide the foundation for an upgraded Ethereum. After two years of testing, the Beacon Chain is finally set to integrate with Ethereum in mid-September — the merge that gives the process its name.</p></div><div><p>As the Merge draws nearer, companies and entrepreneurs with a financial stake in Ethereum mining are growing increasingly concerned. Crypto mining has become a multibillion-dollar business, dominated by publicly traded companies.</p><p>In a recent corporate report, Hive Blockchain, a crypto mining firm that draws revenue from both Bitcoin and Ethereum, <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.hiveblockchain.com/medias/uploads/2022_06-30_HIVE_FS_FINAL.pdf\">said</a> a switch to proof of stake “may render our mining business less competitive or less profitable.” Hive did not respond to requests for comment.</p><p>Some Ethereum miners are fighting back. Mr. Guo, who used to run an Ethereum mining operation in China, has <a rel=\"noopener noreferrer\" title=\"\" href=\"https://www.bloomberg.com/news/articles/2022-08-06/crypto-miner-pushback-suggests-ethereum-offshoots-are-coming?sref=zVYYYI5e\">mobilized</a> a group of engineers who are working on an alternate currency to rival the upgraded Ether.</p><p>Despite the pushback, the Merge appears close to completion. A final test, known as the Goerli merge, was finished successfully this month.</p><p>Mr. Ryan, the Ethereum researcher, said he planned to gather in person with some colleagues to mark the actual Merge, the culmination of five years of his professional life.</p><p>Planning the celebration is complicated. While developers have estimated a Sept. 15 date, the Merge’s precise timing is uncertain and subject to complex technical factors. Any glitch could cause another delay.</p></div><div><p>“This might happen at 4 in the morning. It might happen at 2 in the afternoon,” Mr. Ryan said. “But I’ll definitely be awake.”</p></div></section></article></div>",
  "text": "Credit...Timo LenzenA long-awaited upgrade to Ethereum, the most popular crypto platform, may make the technology more environmentally sustainable. But it comes with risks.Credit...Timo LenzenDavid Yaffe-BellanyDavid Yaffe-Bellany, who covers crypto, reported this article from San Francisco.Aug. 26, 2022The cryptocurrency industry has endured a terrible year. A devastating crash wiped nearly $1 trillion from the market, draining the savings of thousands of people. Several companies filed for bankruptcy.Now the industry is fixated on a potential saving grace: a long-awaited software upgrade to the most popular cryptocurrency platform, Ethereum, which provides the technological backbone for thousands of crypto projects. The upgrade — known as the Merge — has gained near-mythical status after years of delays that left some insiders questioning whether it would ever happen.But if all goes according to plan, the Merge will take place around Sept. 15, more than eight years after it was initially discussed. The change would shift Ethereum to a more energy-efficient infrastructure, addressing the widespread criticism that crypto’s climate impact outweighs its possible benefits. And it would lay the foundation for future upgrades to reduce the hefty fees required to conduct transactions in Ether, the platform’s signature currency and the second-most valuable digital asset after Bitcoin.“This transition is essentially laying a road map to a future which is far more scalable, far more energy efficient and a lot more usable to the normal person,” said Joseph Ayoub, an analyst at Citi who has studied the Merge. “It’s laying the groundwork for adoption.”But the risks are profound. Even by crypto standards, the process is almost ludicrously complicated.For months, insiders have engaged in frantic, jargon-heavy discussions of developments such as the Goerli Testnet Merge and the Beacon Chain’s Bellatrix upgrade, crucial software changes leading up to the main event. A botched Merge would imperil the thousands of crypto applications that operate using Ethereum, which collectively handle more than $50 billion in user funds.“It’s flying the jet, and changing the engine in the sky,” said Chandler Guo, a crypto industry veteran who leads a group opposing the Merge. “It’s very difficult. It’s very dangerous.”ImageCredit...Sarah Pabst for The New York TimesEthereum is a blockchain, a publicly viewable ledger where exchanges of digital coins are recorded. Transactions on the chain are conducted in Ether.The platform was started in 2013 by a teenage programmer, Vitalik Buterin, who is now regarded as one of the industry’s elder statesmen. Mr. Buterin wanted to create a crypto system that was more flexible than Bitcoin and could instantly execute financial contracts and other complex forms of exchange.Ethereum’s design allows it to support a range of financial engineering. Programmers can build applications using the software to perform more complicated tasks than simple money transfers. Thousands of businesses and projects in the experimental world of decentralized finance now use the platform to offer lending, borrowing and other sophisticated investment options. Many nonfungible tokens — the unique digital collectibles known as NFTs — are built on Ethereum.At its core, the Merge is a change to Ethereum’s verification system. When someone sends money in a traditional transaction, a bank serves as the middleman, verifying that one person has sufficient funds to pay someone else.Crypto operates without that middleman. In this alternate financial system, transactions are verified by a scattered network of computers. Anyone can plug a machine into the network by running software that solves complex puzzles, an energy-guzzling process for confirming transactions. Essentially, the computers are racing one another: When the puzzle is solved, the winning participants are rewarded with new coins in the digital currency they are verifying.This verification process is widely known as crypto mining and has the technical name “proof of work.” By some estimates, the amount of energy consumed each year in mining is comparable to the annual emissions of entire countries.The Merge is set to shift Ethereum to an alternate framework called “proof of stake,” which requires less energy. In a proof-of-stake system, computers don’t burn energy racing to verify transactions. Instead, crypto investors deposit a certain number of digital coins in a shared pool, which enters them into a lottery. Each time an exchange happens, a participant is selected from the lottery to verify the transaction and win the rewards.The shift is expected to cut Ethereum’s energy use by more than 99 percent, which crypto boosters hope will make the technology more popular.“The difference in hardware and energy consumption is so significant,” said Preston Van Loon, a developer working on the Merge. “When NFTs were blowing up, people were saying, ‘I’d love to have an NFT, but it feels like I’m burning a forest down.’”ImageCredit...Stacy Kranitz for The New York TimesSwitching to proof of stake could also help solve another of Ethereum’s biggest problems: the substantial fees required to use the network. Ethereum can handle only a certain amount of activity at a time, so when there’s demand for the platform, the price of using it goes up. Anyone sending Ether must pay a “gas fee” — a transaction charge that at times has risen as high as $200.The Merge won’t eliminate that problem immediately, but developers say it will lay critical groundwork for future upgrades designed to minimize fees.But a change on this scale could also make Ethereum vulnerable to hacks or other disruptions, some crypto experts say. “Any time you’re making changes to a complex system, there must necessarily arise unintended consequences,” said Christopher Calicott, a crypto venture investor.Much of the criticism is fueled at least partly by self-interest. Many of the Merge’s most vocal opponents are businesses that have built expensive data centers to mine Ether in the proof-of-work system.The Merge’s origins date to the time of Ethereum’s creation. Mr. Buterin raised the possibility of switching to proof of stake in 2014, but at that point, the system was untested. The most successful cryptocurrency was Bitcoin, which uses proof of work.ImageCredit...Michael Ciaglo/Getty ImagesSince then, many newer cryptocurrencies have used proof of stake successfully. Ethereum programmers have been working in earnest on the switch for at least four years. The work is complex, and progress has been slow. Engineers had to construct a new blockchain, and run tests to check for security holes or other technological bugs that might disrupt the transition.A nonprofit called the Ethereum Foundation helps supervise the platform. But in reality, Ethereum is run by a loose group of engineers across the world; no top-down authority orchestrated the Merge. Periodically, programmers get together on publicly streamed video calls to debate technical aspects of the shift.At one point, the shift to proof of stake was supposed to happen as early as 2016. As the Merge came together this year, crypto enthusiasts expected a June launch date. Then the Merge was pushed back to August. Now it’s set for next month.In the crypto world, the delays became a kind of joke. Early on, engineers installed what’s known as a “difficulty bomb” in Ethereum’s code. It was designed to keep them honest: After a set amount of time, the bomb would go off, causing disruptions in the Ethereum network unless it was switched to proof of stake. But every time the bomb was supposed to go off, the engineers created a new bit of code to defuse it, somewhat defeating the point.“We’re not doing this lightly,” said Danny Ryan, an Ethereum Foundation researcher who has worked on the platform since 2017. “There’s been countless amount of engineering and testing and academic vetting.”In December 2020, Ethereum programmers took a big step toward the Merge by releasing a crypto platform called the Beacon Chain, a proof-of-stake system designed to provide the foundation for an upgraded Ethereum. After two years of testing, the Beacon Chain is finally set to integrate with Ethereum in mid-September — the merge that gives the process its name.As the Merge draws nearer, companies and entrepreneurs with a financial stake in Ethereum mining are growing increasingly concerned. Crypto mining has become a multibillion-dollar business, dominated by publicly traded companies.In a recent corporate report, Hive Blockchain, a crypto mining firm that draws revenue from both Bitcoin and Ethereum, said a switch to proof of stake “may render our mining business less competitive or less profitable.” Hive did not respond to requests for comment.Some Ethereum miners are fighting back. Mr. Guo, who used to run an Ethereum mining operation in China, has mobilized a group of engineers who are working on an alternate currency to rival the upgraded Ether.Despite the pushback, the Merge appears close to completion. A final test, known as the Goerli merge, was finished successfully this month.Mr. Ryan, the Ethereum researcher, said he planned to gather in person with some colleagues to mark the actual Merge, the culmination of five years of his professional life.Planning the celebration is complicated. While developers have estimated a Sept. 15 date, the Merge’s precise timing is uncertain and subject to complex technical factors. Any glitch could cause another delay.“This might happen at 4 in the morning. It might happen at 2 in the afternoon,” Mr. Ryan said. “But I’ll definitely be awake.”",
  "topImage": "https://static01.nyt.com/images/2022/08/23/autossell/still_01_Fin/still_01_Fin-facebookJumbo.jpg",
  "date": "2022-08-26T09:00:22.000Z",
  "keywords": [
    "Ethereum",
    "Vitalik Buterin",
    "Banking and Finance",
    "Computers and the Internet",
    "Tech Industry"
  ],
  "summary": [
    "Ethereum programmers have been working in earnest on the switch for at least four years.",
    "At one point, the shift to proof of stake was supposed to happen as early as 2016.",
    "As the Merge came together this year, crypto enthusiasts expected a June launch date.",
    "Then the Merge was pushed back to August."
  ]
}

Parse anything including Medium.com articles

Example medium article parsed below:https://medium.com/pipfeed/how-does-pipfeeds-recommendation-engine-work-pipfeed-com-2f3c12546fb

00
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
{
  "url": "https://medium.com/pipfeed/how-does-pipfeeds-recommendation-engine-work-pipfeed-com-2f3c12546fb",
  "title": "How does Pipfeed’s recommendation Engine work? — PipFeed.com",
  "author": "Shashank Agarwal",
  "length": 1420,
  "siteName": "PipFeed",
  "description": "At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create…",
  "html": "<div class=\"page\" id=\"readability-page-1\"><div><figure><div tabindex=\"0\" role=\"button\"><p><img role=\"presentation\" src=\"https://miro.medium.com/max/1400/0*ySiYrjZSIUZC_fyT.jpg\" alt=\"\"></p></div></figure><p id=\"5ee3\">At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create a recommendation engine and even hard to measure how well the algorithm is working. Here is how Pipfeed’s recommendation engine works.</p><p id=\"b56b\">The main feed is comprised of “Plugins”. Each of these plugins “competes” to add articles to the user’s feed.</p><h2 id=\"6954\">Pipfeed’s plugins</h2><p id=\"6937\">Pipfeed has a plugin system internally and each of the plugins have different approach to add articles to the user’s main feed.</p><ul><li id=\"bde2\"><strong>Subscriptions:</strong> Add articles from the blogs user has subscribed to</li><li id=\"1e39\"><strong>Interests:</strong> Adds articles from user’s Interests on their profile</li><li id=\"406a\"><strong>Trending:</strong> Adds articles that are trending on PipFeed</li><li id=\"3ed2\"><strong>Past Liked Blogs</strong>: Adds articles from user’s previous most liked blogs</li><li id=\"a6e6\"><strong>Past Liked Interests:</strong> Adds articles from user’s past liked Interests</li><li id=\"268c\"><strong>Collaborative filtering:</strong> Adds articles similar to previously read articles</li><li id=\"ab18\"><strong>Content-based filtering:</strong> Find users with similar reading behavior and then adds from that user’s past history</li><li id=\"2fe3\">And more…</li></ul><h2 id=\"568c\">The Feedback Loop</h2><p id=\"5b8c\">Pipfeed’s recommendation engine re-balances the importance given to each plugin based on user interactions. So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed.</p><p id=\"824c\">So the system keeps “rebalancing” itself over time and as the user uses the app more, the more personalized the feed becomes.</p></div></div>",
  "text": "At Pipfeed we take price in creating a world-class recommendation engine and have spent a lot of time perfecting it. It is hard to create a recommendation engine and even hard to measure how well the algorithm is working. Here is how Pipfeed’s recommendation engine works.The main feed is comprised of “Plugins”. Each of these plugins “competes” to add articles to the user’s feed.Pipfeed’s pluginsPipfeed has a plugin system internally and each of the plugins have different approach to add articles to the user’s main feed.Subscriptions: Add articles from the blogs user has subscribed toInterests: Adds articles from user’s Interests on their profileTrending: Adds articles that are trending on PipFeedPast Liked Blogs: Adds articles from user’s previous most liked blogsPast Liked Interests: Adds articles from user’s past liked InterestsCollaborative filtering: Adds articles similar to previously read articlesContent-based filtering: Find users with similar reading behavior and then adds from that user’s past historyAnd more…The Feedback LoopPipfeed’s recommendation engine re-balances the importance given to each plugin based on user interactions. So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed.So the system keeps “rebalancing” itself over time and as the user uses the app more, the more personalized the feed becomes.",
  "topImage": "https://miro.medium.com/max/1024/0*ySiYrjZSIUZC_fyT.jpg",
  "date": "2020-11-11T05:22:00.807Z",
  "keywords": [
    "recommendation engine",
    "Artificial Intelligence",
    "Engineering",
    "Computers and the Internet",
    "Tech Industry"
  ],
  "summary": [
    "Here is how Pipfeed’s recommendation engine works",
    "The main feed is comprised of “Plugins”.",
    "Each of these plugins “competes” to add articles to the user’s feed",
    "So we measure articles from which plugin is the user most interacting with and that plugin gets to put more articles in the user’s feed"
  ]
}

Built by ex-AWS engineer

The API is built to scale, handles edge cases,
completely secure, and is anonymous. We never store anything on our servers.

Agon

1. Extract all the fields

We return not just the title, and body but also the author, summary, tags, and much more.

Agon

2. AI powered

We use our in-house trained AI models to predict the category & generate a summary.

Agon

3. Scalable

Did your app go viral? Don’t worry! The API scales flawlessly to millions of API calls.

Pipfeed
One API, millions of use cases

Avoid months of development time building your custom URL extractor

We have spent months fine-tuning each small part of our article extract API so you don't have to. One API can help you achieve various use cases. These are some of the use-cases our customers Pipfeed's API help enable:

Add "Reader Mode" to your apps.

Send "summarized" content to your readers.

Use parsed data in training AI models.

Allow readers to download the content and read it offline.

Choose the Plan that's
Best For You

$0/ Month

Basic

Best for trying out APIs for free.

  • 100 APIs/month
$9/ Month

Pro

For small apps & usecases

  • 5,000 API calls per month
  • 7 day caching
  • 1 Request per second max
$19/ Month

Ultra

For apps with over 1000 users

  • 10,000 API calls per month
  • 7 day caching
  • 1 Request per second
  • AI generated Summary
  • AI Predicted Categories
$49/ Month

Mega

For apps that need scale.

  • 50,000 API calls per moth updates
  • 7 day caching
  • 5 Requests per second
  • AI generated Summary
  • AI Predicted Categories

Frequently asked questions

Feeling inquisitive? Have a read through some of our FAQs or contact our supporters for help

Customized Plans

Don't see a plan that suits your needs? Please feel free to contact us.

Have any questions?

Feel free to send us a message if you have any queries.

You can use our APIs by signing up on RapidAPI

If you need any help, please feel free to email us at "contact[@]pipfeed.com"

We offer all types of customization based on our customer's requirements. Please send us an email at: "contact[@]pipfeed.com"

Depending on your plan's specifications, you will either incur overage charges or be suspended.

We charge your credit card upon subscription to an API's plan and at the next recurring interval.

For refund requests, please contact us at [email protected]

Pipfeed Inc is Delaware incorporated LLC.