Now a much better way to do this is to use the rdd.aggregateByKey() method. Because this method is so poorly documented in the Apache Spark with Python documentation — and is why I wrote this Q&A — until recently I had been using the above code sequence. But again, it’s less efficient, so avoid doing … Read more
Try this: SELECT SEC_TO_TIME(AVG(TIME_TO_SEC(`login`))) FROM Table1; Test data: CREATE TABLE `login` (duration TIME NOT NULL); INSERT INTO `login` (duration) VALUES (’00:00:20′), (’00:01:10′), (’00:20:15′), (’00:06:50′); Result: 00:07:09
If I understand what you need, try this: SELECT id, pass, AVG(val) AS val_1 FROM data_r1 GROUP BY id, pass; Or, if you want just one row for every id, this: SELECT d1.id, (SELECT IFNULL(ROUND(AVG(d2.val), 4) ,0) FROM data_r1 d2 WHERE d2.id = d1.id AND pass = 1) as val_1, (SELECT IFNULL(ROUND(AVG(d2.val), 4) ,0) FROM … Read more
Here’s a short and sweet solution (perhaps not the fastest though). It takes the difference between each date in the date list and some arbitrary reference date (returning a datetime.timedelta), and then sums these differences and averages them. Then it adds back in the original reference date. import datetime def avg(dates): any_reference_date = datetime.datetime(1900, 1, … Read more
If your array arr has a length divisible by 3: np.mean(arr.reshape(-1, 3), axis=1) Reshaping to a higher dimensional array and then performing some form of reduce operation on one of the additional dimensions is a staple of numpy programming.
Using an enhanced for would be even nicer: int sum = 0; for (int d : data) sum += d; Another thing that will probably give you a big surprise is the wrong result that you will obtain from double average = sum / data.length; Reason: on the right-hand side you have integer division and … Read more
With Java 8 it is a bit easier: OptionalDouble average = marks .stream() .mapToDouble(a -> a) .average(); Thus your average value is average.getAsDouble() return average.isPresent() ? average.getAsDouble() : 0;
In analogy to the cumulative sum of a list I propose this: The cumulative average avg of a vector x would contain the averages from 1st position till position i. One method is just to compute the the mean for each position by summing over all previous values and dividing by their number. By rewriting … Read more
EDIT: It seems that mov_average_expw() function from scikits.timeseries.lib.moving_funcs submodule from SciKits (add-on toolkits that complement SciPy) better suits the wording of your question. To calculate an exponential smoothing of your data with a smoothing factor alpha (it is (1 – alpha) in Wikipedia’s terms): >>> alpha = 0.5 >>> assert 0 < alpha <= 1.0 … Read more
As numpy.convolve is pretty slow, those who need a fast performing solution might prefer an easier to understand cumsum approach. Here is the code: cumsum_vec = numpy.cumsum(numpy.insert(data, 0, 0)) ma_vec = (cumsum_vec[window_width:] – cumsum_vec[:-window_width]) / window_width where data contains your data, and ma_vec will contain moving averages of window_width length. On average, cumsum is about … Read more