| stevehopwoodforex.com https://www.stevehopwoodforex.com/phpBB3/ Print view |
|
| Data Project https://www.stevehopwoodforex.com/phpBB3/viewtopic.php?t=923 |
Page 1 of 1 |
| Author: | boldtrader [ Thu Nov 01, 2012 5:57 pm ] |
| Post subject: | Data Project |
Hello fellow coders. I want to say thank you to Steve for the great forethought of developing this great service and the dedication it requires to continue its operation and development. The work(s) you provide here is greatly appreciated. Hopefully this great community of resources can lead me in a positive direction with some assistance and potential solutions. I am involved in a "Data Project" of analyzing historical price data of multiple time periods vs indicator values output of the same price data vs historical database of "events" for the FX. I have scripts which will create the csv files for the price data and indicator data from Empty4. I have the csv file of the events. For now, my focus is on the Forex market, using the Empty4 platform. Real time connectivity from Empty4 to the database will be needed. I am aware of a DLL which may be able to provide this connection, as well as, the DDE to MS XL which can then be setup for auto import to MYSQL. In either process, I am not sure of the direction to take. I am a market participant of 16+ years with a high degree of experience trading, algo development, strategy development, managed funds and various other aspects of the Securities Brokerage Industry including back office Trade Desk operations. I am not a programmer. I am self taught MQL4 and now learning Python, R, and C++. I have been looking at software such as Hadoop, Cassandra, Tableau and MySQL for possible solutions. I have also considered several other software programs which seem to provide part of the solution for the real time queries and real time delivery stream. I am like a sponge, learning as much as I can, but again.. I am not a programmer. I am looking for guidance and direction of possible software solutions. I have no idea where to begin or how to begin. I am willing to learn or at least surround myself with individuals more knowledgeable than I. I am not seeking back testing results. I am very familiar with how Empty4 works, its flaws in backtesting and how to obtain 99% modeling quality. My "Data Project" consists of continuously merging price data for a large number of financial instruments with proprietary indicator data, "event data", other data (based on user selected input analysis option filters). My goal is to merge all of this data into a format which can provide analystic queries into a distributed, real-time computation system in a real time delivery stream processing format. Additionally, the project should have the ability to use current data results, merged with historical data results (the complete statistical results compilation) to form new, updated statistical results with probabilistic tendencies based on the merged statistical results. I know this can be done. I don't know the details or the process by which this can be accomplished. Thank you for taking time to read this call for assistance. I hope it is not misplaced by being in this forum. If so, please forgive my enthusiasm to reach out to someone who is more knowledgeable than I to develop this project in a short time. |
|
| Author: | jcl [ Thu Nov 01, 2012 6:32 pm ] |
| Post subject: | Re: Data Project |
The critical point seems to be the real time query ability. SQLite is relatively fast, but you need to organize the database well for generating fast responses. Maybe you don't need a database at all - this depends on the complexity of the queries. MQL4, Python, and R are all interpreted languages and too slow for time critical applications, so I would use C++ for the project. |
|
| All times are UTC | Page 1 of 1 |
|
Powered by phpBB® Forum Software © phpBB Limited |
|