Hacker News Reader

Comment by megaframe | Hacker News Reader

Combination Machine Learning, NLP, Neural Networks, stock analyzer. Does automated day trading. Not HFT style stuff. More what a bank pays an analyst to do, type of trading, but smaller scale since it's only my personal funds.

View on HN

This is really interesting, I would love to hear more about how you are going about doing this.

There's a whole world of people doing this. Some even package it up into solutions to sell you as though it's this black box solution you can plop down and just make money out of it. The combination layers of machine learning and quirkiness of the market, often require some small hand holding, and if you venture into certain areas of the market HFT's and other traders smarter than you will take advantage of your algorithmic trading by placing orders ahead of you.

Point is, it's not without some heavy risks, and regular attention.

I am working on something similar for forex. Did you use an existing neural network library for this or make your own?

Also care to share any pitfalls you fell into at the start? Right now I am working on building modules with the purpose of collecting data for training.

I started with the open NLP packages, but I rewrote pretty much everything, including my own ML algos. Typically based on the open source one. I kept running into scaling and use case performance issues. On some occasions things I did turned out to be solutions I'd find buried in the language of some google paper, which is mildly frustrating.

Forex is a tricky space, volatility and fees can really funk you up. I do a little there because the returns tend to justify it. Be wary of smaller and or unknown names. Manually check out anything you allow your algos to trade... at least till you come up with a comprehensive set of rules you can trust.

I tend to stick to US equities. Just because my risk tolerance is low for what amounts to a side project using my personal funds. Which I'm often not looking at until all the trading is done for the day.

I am curious where do you get your data feed for trading. Do you pay for the data or are you using a free api? The main i reason i choose forex was due to ease of API use for data and trading.

Also follow up question, what sort of inputs are you using for training. Are you using mainly sentiment and price history or mostly technical indicators?

Level I and Level II data from my broker's API (Interactive Brokers). It's a free service as long as you have enough trading fee's every month. TDAmeritrade has an OK API as well. Both services have some limitations on data feeds (number of symbols you can actively review). There's some lag in data (sub what a human could notice) but again these are essentially free, and unless you're trying to be an HFT or scraping stocks this shouldn't matter. Neither of those activities are recommended, the people you'd be going up against have a lot more money and much better resources. News data is aggregated from a variety of sources and paid rss news feeds.

Yes to sentiment, yes to history (though I keep my own history for the sake of granularity and not paying twice for data), and sort of for technical indicators... technical indicators are helpful for humans, but if your ML algo is using a data set that spans a multi-day window already a technical indicator of the direction of movement in the last few days is just telling your algo something it already knows.

I am surprised to hear that your placing less importance on technical indicators. I have been going under the assumption that those should be my main focus for input as they can be easily calculated and translated into simple number inputs.

My thought process is that the more indicators i can feed in with some history the higher the possibility that unique patterns are discovered. I originally thought that sentiment and news would be to complex to break down into a simple input for the network. I want to include them but i was going to place allot less emphasis on them.

Are you simply breaking down sentiment as good vs bad by analyzing text or are you doing something more complex?

I am sorry to be asking so many questions, but i have literally just started working on this for less then a week and I find your success very interesting. At this stage i am treating this as a long experiment.

I got stock data from Yahoo to generate these reports. It is extremely fast and reliable

Have you considered using your bot in the cryptocurrency market? There are no high-frequency traders to worry about because the technology has not reached that level yet. Volatility is high so your algos won't be sitting on the sidelines as much.

Would you mind putting an email in your profile? I'd like to contact you privately.

How much profit does this generate, if I may ask? And how much do you have invested as capital for it to trade on?

I make about what a Hedge fund makes 15-28% a quarter depending on how well the market is doing. I just don't pay hedge fund fee's. It's not wildly innovative I'm just circumventing paying a hedge fund, which often use the exact same methods I used.

Can you elaborate? Any resources on getting started with this kind of thing?

Like most tasks start by teaching yourself to trade on news/price info. Then automate yourself step by step. For instance News articles pop up, what are you trying to glean from them, its usually sentiment and and critical alerts. You can automate that with an NLP. Kind of just goes from there back into higher level decision making processes.

Is there a reason this isn't scaled out as a service?

Yeah, it seems like he could make much, much more.

SEC rules. To do that you need become a trading advisor (similar to a hedge fund), which requires that you have made over $200k/yr for the last 3 years or that you have $2.5Mil in your own assets.

While I've done consistently well so far, I've only been at it full force a few years, and I started with my own cash which isn't nearly that much.

The followup is usually turning this into a business and getting investors... if you looked at my resume not much would make you think I have the skill set for this (though I've spent 3 years building this on my spare time). I'm also not sure I'm really up for the whole start-up thing, I enjoy my somewhat relaxed work schedule/hours. I value my personal life more than this stuff.

I'm open to suggestion on that last one but it's not something I've done before.

Can you contact me (check my profile for email). This is fascinating and I would love to discuss it further but communication via hackernews is pretty limited.

Reply on news.ycombinator.com