How to Screen Thousands of Stocks Without a Data Provider
Stock screeners feel like something you should have to pay a data vendor for. Market cap, sector, P/E, dividend yield, RSI, daily change, all of it across thousands of tickers, updated through the day. In practice a lot of that is one keyless JSON call away, and you can run it yourself. TradingView's screener page is backed by a public endpoint that takes a filter and returns matching symbols: POST https://scanner.tradingview.com/america/scan The body is just a filter, the columns you want, a so
Stock screeners feel like something you should have to pay a data vendor for. Market cap, sector, P/E, dividend yield, RSI, daily change, all of it across thousands of tickers, updated through the day. In practice a lot of that is one keyless JSON call away, and you can run it yourself.
The scanner most people scroll past
TradingView's screener page is backed by a public endpoint that takes a filter and returns matching symbols:
POST https://scanner.tradingview.com/america/scan
Enter fullscreen mode Exit fullscreen mode
The body is just a filter, the columns you want, a sort, and a range:
{
"filter": [
{ "left": "market_cap_basic", "operation": "egreater", "right": 10000000000 },
{ "left": "dividend_yield_recent", "operation": "egreater", "right": 3 }
],
"columns": ["name", "close", "dividend_yield_recent", "market_cap_basic", "RSI", "sector"],
"sort": { "sortBy": "market_cap_basic", "sortOrder": "desc" },
"range": [0, 100]
}
Enter fullscreen mode Exit fullscreen mode
You get back a totalCount and a data array, one entry per symbol, with a d array of values lined up to the columns you asked for. No login, no API key, no headless browser. The same endpoint serves other markets too: swap america for crypto, forex, india, uk, and more.
Push the filter down, do not pull the table
The mistake I see most often is pulling a giant table and filtering it in your own code. That is slow, and it ships data you never wanted. The scanner filters server side, so you only receive the rows that already match. Want large cap names that are oversold? Send market_cap_basic greater than ten billion and RSI less than thirty, and that is all that comes back.
A few useful column ids, since they are not obvious:
-
closeis the last price -
changeis the percent change on the day -
market_cap_basicis market cap -
price_earnings_ttmis trailing P/E -
dividend_yield_recentis the dividend yield -
Perf.YTDis year to date performance
Operations you will reach for: egreater and eless for greater or equal and less or equal, in_range for a band or a set of values, and greater and less for strict bounds. in_range doubles as a set membership test, so a list of sectors works the same way as a numeric band.
Paging and cost
Each request takes a range, so paging is just moving the window: [0,100], then [100,200], until you hit totalCount or the cap you set. Because there is no browser and no proxy, a full screen of a few hundred symbols costs almost nothing to run. Normalize the numbers once when they come in and every downstream join behaves.
I packaged it
I turned this into a small Actor on Apify so it is callable from code without writing the request by hand. You set friendly filters like market cap, sector, P/E, dividend yield, RSI or daily change, and it returns one tidy row per symbol with price, change, volume, market cap, P/E, dividend yield, RSI, sector, industry, exchange, country and YTD. It is the newest of a growing set of keyless finance scrapers I have been shipping. The first rows of every run are free so you can check the output before you scale up.
The wider point stands on its own though: before you reach for a paid data plan, check whether the site you already trust is serving the same numbers over a public endpoint. Often it is.
