This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public <T> T streamQuery(String sql, Function<Stream<SqlRowSet>, ? extends T> streamer, Object... args) { | |
return jdbcTemplate.query(sql, resultSet -> { | |
final SqlRowSet rowSet = new ResultSetWrappingSqlRowSet(resultSet); | |
final boolean parallel = false; | |
// The ResultSet API has a slight impedance mismatch with Iterators, so this conditional | |
// simply returns an empty iterator if there are no results | |
if (!rowSet.next()) { | |
return streamer.apply(StreamSupport.stream(Spliterators.emptySpliterator(), parallel)); | |
} | |
Spliterator<SqlRowSet> spliterator = Spliterators.spliteratorUnknownSize(new Iterator<SqlRowSet>() { | |
private boolean first = true; | |
@Override | |
public boolean hasNext() { | |
return !rowSet.isLast(); | |
} | |
@Override | |
public SqlRowSet next() { | |
if (!first || !rowSet.next()) { | |
throw new NoSuchElementException(); | |
} | |
first = false; // iterators can be unwieldy sometimes | |
return rowSet; | |
} | |
}, Spliterator.IMMUTABLE); | |
return streamer.apply(StreamSupport.stream(spliterator, parallel)); | |
}, args); | |
} |
fun queryStream(sql: String, converter: (SqlRowSet) -> T, args: Array): Stream {
val rowSet = jdbcTemplate.queryForRowSet(sql, *args);
...
Meh, that's totally missing the point!
The idea is lazy consumption (on the fly row processing), but with JdbcTemplate::queryForRowSet
you're just aggregating results in memory! For that to workJdbcTemplate
would have to keep the db connection open after making that call and it just doesn't do that.
You can replace your whole snippet with jdbcTemplate.queryForList(...).stream()
to get the same result.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks for putting me on the right track with this. I am using this from Kotlin and came up with a Streaming version that works fine if you use it in a transaction. Also, you have a bug in your exit condition and you are skipping the last row. I fixed this by making hasNext do most of the work here (a good idea when implementing iterators) and by streaming the converted rows and not the rows themselves. This also gets rid of the problem you have with the first row.
This is Kotlin but you should be able to convert this back to Java easily. I use JdbcTemplate together with a TransactionTemplate to make sure the connection stays open while I stream the results.