On Sat, Feb 21, 2015 at 12:25:53AM +0900, Akira Hayakawa wrote:
> Yes.
How big are your log chunks? Presumably they're relatively small (eg,
256k). In which case you can optimise for the common case where you
have enough bios to hand to build your log chunk by just referencing
the bio payloads, rather than copying. It's only the last bit of io
in a burst that should be using this copying slow path.
- Joe
The size is configurable but typically 512KB (that's the default).
Refer to bio payload sounds really dangerous but it may be possible
in some tricky way. but at the moment I am not sure how the
implementation would be.
Is there some fancy function that is like memcpy but actually "move"
the ownership?
- Akira
On 2015/02/21 0:50, Joe Thornber wrote:
> On Sat, Feb 21, 2015 at 12:25:53AM +0900, Akira Hayakawa wrote:
>> Yes.
> How big are your log chunks? Presumably they're relatively small (eg,
> 256k). In which case you can optimise for the common case where you
> have enough bios to hand to build your log chunk by just referencing
> the bio payloads, rather than copying. It's only the last bit of io
> in a burst that should be using this copying slow path.
>
> - Joe
On Sat, Feb 21, 2015 at 01:06:08AM +0900, Akira Hayakawa wrote:
> The size is configurable but typically 512KB (that's the default).
>
> Refer to bio payload sounds really dangerous but it may be possible
> in some tricky way. but at the moment I am not sure how the
> implementation would be.
>
> Is there some fancy function that is like memcpy but actually "move"
> the ownership?
When building up your log chunk bio copy the bio_vecs (not the data)
from the original bios. You can't complete the original bios until
your log chunk has been written.
- Joe
To be clear, bio's semantics doesn't require a io is written on
persistent medium before any ack. The border line is that ios that's acked
are persitent before ack to REQ_FLUSH request.
So, writing on the volatile buffer (log chunk in this case) and then ack
is safe if the data gets persistent before some future REQ_FLUSH
request is acked. That's dm-writeboost does.
And in general, ack should be quick as possible otherwise may incur
some problems such as upper layer may suspend any other requests.
The bio_vecs solution works only for a tiny prototype.
If I apply the solution there will appear the following problems
1. The write to the cache device isn't one single write.
This causes atomicity problem. And may cause performance
degradation.
2. We need to compute checksum of the entire log chunk before write.
Without this, the user isn't safe from partial write problem.
Like the 1 above, atomicity is to be cared.
(btw, I don't think dm-cache that has separete data device and
metadata device can guarantee this level of safetiness)
3. Don't ack any bios until the full buffer is written is harmful.
We should ack as quick as possible as explained above.
4. Read caching becomes infeasible. It needs copying of the read data.
My conclusion is write buffer in practice should be a single buffer and
copying is inevitable.
>From a engineering point of view, memory copy can't be the bottleneck
(before that, SSD's throughput hits) so we shouldn't hack for the little
improvement.
- Akira
On 2015/02/21 1:17, Joe Thornber wrote:
> On Sat, Feb 21, 2015 at 01:06:08AM +0900, Akira Hayakawa wrote:
>> The size is configurable but typically 512KB (that's the default).
>>
>> Refer to bio payload sounds really dangerous but it may be possible
>> in some tricky way. but at the moment I am not sure how the
>> implementation would be.
>>
>> Is there some fancy function that is like memcpy but actually "move"
>> the ownership?
> When building up your log chunk bio copy the bio_vecs (not the data)
> from the original bios. You can't complete the original bios until
> your log chunk has been written.
>
> - Joe