I’m looking for a nocode plateform that could allow me to auto pull/retreive data from online datasheets.
We are facing a problem with this Gps data provider not offeing Direct or API DB access.
So far, we can download excell files of sheets that we can generate only after navigating through menu, selecting Start/end Calendar dates, Checking box for needed vehicle, etc, etc…this is tedious to repeat every week, but only doing such can we produce a display view of some look alike sheet.
We are wondering if Parabola can help in :
1 - automating all this process of navigating/generating the View before downloading the sheet.
2 - Creating some dashboard with reports of auto-calculated records based on the excell sheet…so indeed, we would RATHER need the processing to be done ALL online instead of downloading an excell…and reloading the reports…
…is the automated Url/Menu raw DATA pulling possible with Parabola so far ?
Thanks for any suggestions.
joined : detail of selection STEPS one has to go through to obtain a view (equivalent to the Excel sheet to be generated - *visible on the right side)
Sounds like a really interesting use case. Before we dive any further, can you share what platform you’re trying to integrate with? To answer your questions:
Parabola may be able to generate views before downloading the sheet. Do they offer any sort of API integration for that? I know you mentioned there is no database access via API.
Same thing here. If your service has an accessible endpoint to send data to, you could use the “Send to an API” step to help generate your dashboards.
Lastly, would you be able to clarify what you mean by asking if automated URL/Menu raw data possible? Some download links can be inserted into the “Pull from an API” step. You can insert the URL as the API Endpoint URL.
Keep me posted, and I’d be happy to point you in the right direction.
So…as for what platform I am trying to integrate with…;if we are talking about the source side of this database, its not a “known” platform like Gdrive or else…its just a GPS maker website http://ms03.trackingmate.com/main.aspx# with no known platform that I could tell you about.
All I can see is that they are using aspx webpages.
Else, let me know if I didn’t get the question right ?
no API, no nothing…as I mentioned, in order to see some actual data, one has to go to several menus, clicking here and there, selecting dates then button “search”…and only then it will produce some view of the data you can see behind. you can also select from popup menus data for the last 3,5,15 days and so on…if this what you want to…
Then once this step is done, you now have to click again on button Download and this will allow you to download an excell file of your customized generated view (to your PC HD…)
about the “Automated URL raw data retreive”, this would be us “programing” some scenarion like :
On every Mondays - Go to this Url http://“gps blabla”…then “press select button” : All, then enter date starting = xxxx , date ending yyy, then press button search…etc… and have the scenario running this sequence of instructions to produce the latest updated view…
A second scenario following would be also created to have the above create “view.excel” result (that we cant actualy grab from any folder) being downloaded to some folder on the parabola server - or converted to a parabola temporary table, then run predifined filters on it so as to produce the report online…without ever needing to download the file on some local PC.
the excel file can only be downloaded using a button, we have no access to a folder view of it.
Both scenari would be automated and repeated on every mondays without us manulay inputing anything after they are created.
So its kind of programing a robotic system …that can virtualy Login and send clicks to a webpage since - how you can realize - we have no way of retreiving this data using a simple SQL query !
Parabola cannot be able to perform any automated menu clicks. There may be another option worth pursuing.
When you download the CSV or Excel file after configuring your settings, are you able to see the URL of the download link? Would you be able to share it with me?
Take the following URL as an example: https://gps.com/download/some-file.csv?start=xxxx&end=yyyy
Permitting Parabola can import that data, we could use a “Start with date & time step” alongside an “Enrich with an API” step to dynamically insert the date range for the last 5 days, 10 days, etc.
This example shows the column creation of the date range we want:
I’m thinking that retreiving data using any non-manual method is just going to be impossible with this provider which seems to be locking us (on purpose forcing us to use his own unpractical dashboard)…
So i’m kind of forgetting about finding any method for this part of the process…meanwhile…if we have to keep on using the ageless manual way to download an excell file, we are likely to keep doing so. That said… we are now focussing on setting a scenario to process the excell file we would then upload manualy to the platform :
The focus would be on creating filters on the ON/OFF swithing engine column so each business can keep an eyes on what days their vehicle’s fleet were used (GPS equipement are collecting many genral info beside jsut a GPS coordinate).
So a global output report for each Biz owner would likely, sort each vehicle plate number grouping it by “On then Off state” on each day of the week, the idea behind is to send (pdf/email) a quick view report of vehicle ordered by day of the week that a supervisor can quickly check to discover any missuse of a vehicle. Nothing very demanding…Though…we are wondering how would parabola deal with the uploading of a large excell file… ?
That is, how many excell file lines/recods can the server process everyday if our vehicle fleet includes hundreds of vehicles…what are your server ressource limitations…?
Would Parabola offers some way to selectively filter the “interesting” records during a file upload ?
That is, among 100 lines, there is probably one 2 lines refering to the On and Off engine TIME status…the other 98 lines would be useless in the process since they will not mention any change of satus…Thus if we can only filter these while uploading, the uploaded DATA to process would be faster and much smaller…(just an idea)
As for your given URL sample, there is nothing such we can use due to the non transparent undisclosed query system being used by the GPS company. *We know how to run such GET/POST queries against a regular DB.
Looking for some helpfull info !
Thks
PS: not sure how to send you a sample excel file ?
Good questions. It sounds like using uploading an Excel file is the best option moving forward. Here are some of the performance limits to take into consideration:
Max file-size: ~200MB
Max row count: 5 million
Max cell count: 150 million
Max column-count: 5000
Using a “Filter rows” step, Parabola can filter for the ON/OFF time status as long as that is in a separate column. Just note, the filtering would happen after uploading your file to Parabola.
Hopefully, our performance capabilities should be able to handle your large file uploads. Try uploading a file and see if Parabola can import and parse the data!
Thanks for the numbers, this will help us define our upload/queries !
yep I’ve uploaded a file yesterday and working on some filtering setups…
*if you have some smart settings to achieve the following, I d be interested to see how it could solve the problem on the line marked as “looking for” - (On/Off status is indeed stored in its own column !)
I’m looking for those steps :
Grouping each vehicle ID in ASC order - Done !
Removing Blank On/off status lines - Done !
Display by (Vehicle ID and then) Date-Time order - Done !
Split a Vertical List into a 7 days column list >>> Looking for
(the idea is listing vehicles on-off status in a Horizontal layout that order them into each (DAY) column from Monday<->Sunday )…if that is doable.
Display the line stating about an “Engine On” Vehicle ID Followed by the VERY NEXT time its “Engine Off” status then appears (in the following hour/s most likely)… <<<< Looking for.
Clarifying that : there is some “cleaning” needed with some redundant “Engine Off” lines.
The GPS equipement keeps on sending (unecessary Engine-Off) status recordings after the time it efectively happened.
.so we want to filter or hide ANY unecessary “Engine Off” status and - for each vehicle - only keep the very one following an Engine ON status !!! that’s where stuff complicates
So we have tried achieving that step by using only filters blocks - but is seems one can not reach that goal through only filters…and that it may requires some other formula.
I’ll be looking today if there is another smart way to do that using some “formulas” in Parabola.
Any smart idea shortcut is welcomed.
Number of “User Login” available.
Among our goal : use of your plateform to offer at least 1 Dashboard credential access to 1 user for each of our oresent Client’s fleet; though we have businesses with dozens of vehicles we have much more client with a handfull of vehicles.
Question : Is there anyway to get some greater number of connected users over your Plan’s maximum users limit ?
This would allow us offering the Dashboard to many more potential small clients…
To split a vertical list into multiple columns, try using the “Pivot columns” step. Your pivot column would be your Day column and your value column would be your On/off status column.
Are you able to share an example of your data structure? It may be easiest to recommend a solution to get your rows to represent the on/off time in sequential order.
If it’s easiest, feel free to send an email to help@parabola.io and we can go over this together. We can also chat about your future use cases for Parabola and see if we can find a plan that accommodates the number of users you will need.
We do have a “Count by group” step that can give you a count of the Engine On status per ID. We also have minimum, maximum, average, and sum by group capabilities as well.
In this example, I am counting the unique values across the ID and Engine Status columns using a “Count by group” step:
Ok, yes, I have tested that, but doesnt seem to help in my case.
I was looking for a Groupby function more like to “hide” unwanted repeated “Motor Off” status between 2 “motor On” status…
In that way :
7:00AM On
8:00AM On
9:00AM On
10:00AM On
11:00AM Off
12:00AM On
…
5:00PM Off
6:00PM Off
Would become:
7:00AM On
11:00AM Off
12:00AM On
5:00PM Off
…that’s the final data layout we’d need to present to owners
( The 8,9,10,11AM Off status lines…merge/hide into the “maximum” or Last “Hour column” Line among them, with an “ON” value).
(similar for lines with Off values…merging into “minimum” one)
No eventual Total or count calculation necessary.
Just a merge to hide lines…That is the GroupBy function I was think of.
if not …I guess some LOOP feature would be required to run against the List for a truck can repeatedly turn its engine On and Off durring the day !
Great, looks cool - let me run a check with the original xcel file to make sure we get all on/off status listed…
Thanks for your time,
will keep you updated !
