| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Makes them work like a bool.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
It would segfault before because no files got added to replay
against. Handle this more gracefully and ensure that fio exits
with a non-zero exit status.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The blktrace format is stored in the native endianness of
the machine it is run on. So to reply traces on a machine
with a different endianness, we need to swap the trace
fields. Detect and do this automatically.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
O_ATOMIC makes writes atomic, meaning that they are fully stable
on media (in the event of a power cut) when acknowledged by the
device and OS.
This only truly works on Linux with the pending patches to
add O_ATOMIC.
Updated by Jens to:
- Add man page and HOWTO description of the option
- Make O_ATOMIC imply O_DIRECT, so that it actually works if you
don't set O_DIRECT manually.
- Add the option to the conversion list so it works for
client/server.
- Error handling so that if atomic=1 is set and the OS does not
support it, error out instead of just pretending it works.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
gcc 4.9 rightfully complains about it, though in a very obtuse way:
CC cconv.o
In file included from thread_options.h:5:0,
from cconv.c:3:
cconv.c: In function 'convert_thread_options_to_cpu':
os/os.h:197:16: warning: iteration 2u invokes undefined behavior
[-Waggressive-loop-optimizations]
__le32_to_cpu(*__val); \
^
os/os.h:176:28: note: in definition of macro '__le32_to_cpu'
#define __le32_to_cpu(x) (x)
^
cconv.c:78:17: note: in expansion of macro 'le32_to_cpu'
o->rwmix[i] = le32_to_cpu(top->rwmix[i]);
^
cconv.c:63:2: note: containing loop
for (i = 0; i < DDIR_RWDIR_CNT; i++) {
^
In file included from thread_options.h:5:0,
from cconv.c:3:
cconv.c: In function 'convert_thread_options_to_net':
os/os.h:209:16: warning: iteration 2u invokes undefined behavior
[-Waggressive-loop-optimizations]
__cpu_to_le32(*__val); \
^
os/os.h:179:28: note: in definition of macro '__cpu_to_le32'
#define __cpu_to_le32(x) (x)
^
cconv.c:372:19: note: in expansion of macro 'cpu_to_le32'
top->rwmix[i] = cpu_to_le32(o->rwmix[i]);
^
cconv.c:352:2: note: containing loop
for (i = 0; i < DDIR_RWDIR_CNT; i++) {
^
In file included from thread_options.h:5:0,
from cconv.c:3:
cconv.c: In function 'fio_test_cconv':
os/os.h:209:16: warning: iteration 2u invokes undefined behavior
[-Waggressive-loop-optimizations]
__cpu_to_le32(*__val); \
^
os/os.h:179:28: note: in definition of macro '__cpu_to_le32'
#define __cpu_to_le32(x) (x)
^
cconv.c:372:19: note: in expansion of macro 'cpu_to_le32'
top->rwmix[i] = cpu_to_le32(o->rwmix[i]);
^
cconv.c:352:2: note: containing loop
for (i = 0; i < DDIR_RWDIR_CNT; i++) {
^
In file included from thread_options.h:5:0,
from cconv.c:3:
os/os.h:197:16: warning: iteration 2u invokes undefined behavior
[-Waggressive-loop-optimizations]
__le32_to_cpu(*__val); \
^
os/os.h:176:28: note: in definition of macro '__le32_to_cpu'
#define __le32_to_cpu(x) (x)
^
cconv.c:78:17: note: in expansion of macro 'le32_to_cpu'
o->rwmix[i] = le32_to_cpu(top->rwmix[i]);
^
cconv.c:63:2: note: containing loop
for (i = 0; i < DDIR_RWDIR_CNT; i++) {
^
Fix it by extending the rwmix indexes to the full rwdir count,
even though it doesn't support trim as part of the mix (yet).
Reported-by: Bruce Cran <bruce@cran.org.uk>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Apparently some Solaris' require this for threadsafe
errno.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
If we fail queueing a read or a write, use aio_error() to
retrieve the right error value. This fixes an issue on
(at least) Solaris where we get EAGAIN due to system
shortage of resources, but treat that as a random type
of error due to using errno.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
When run in server mode, jobs are processed before fio_time_init()
is called, which causes pthread_setspecific() to be called before
pthread_key_create(). Work around this by calling it in thread_main()
too.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The network code depends on writev(), so implement it for Windows.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
MinGW has been updated: this causes problems due to changes in header
files, including COM stuff being pulled in resulting in 'interface' being
a reserved keyword, and changes to a few POSIX definitions.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Castor Fu <castor@alumni.caltech.edu>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
In client/server mode, this adds support for json mode. Each job's
details are gradually accumulated and upon completion the client
dumps the full json tree. The tree is annotated with the server
host and port.
Signed-off-by: Castor Fu <castor@alumni.caltech.edu>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Bruce reports:
It seems there's a bug in the x86_64 version of do_cpuid() that causes
fio to segfault when built with clang at lower optimization levels:
static inline void do_cpuid(unsigned int *eax, unsigned int *ebx,
unsigned int *ecx, unsigned int *edx)
{
asm volatile("cpuid"
: "=a" (*eax), "=b" (*ebx), "=r" (*ecx), "=d" (*edx)
: "0" (*eax), "2" (*ecx)
: "memory");
}
via Tijl Coosemans:
Should be: "=c" (*ecx)
But you can also use the '+' modifier and remove the input operands:
: "+a" (*eax), "=b" (*ebx), "+c" (*ecx), "=d" (*edx)
:
: "memory"
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Hong Zhiguo <zhiguohong@tencent.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The child segfault test should catch it, however it does not on
AIX at least.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Akash Verma <akashv@google.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
If the ioengine specific parameter is given on the command line,
unrelated value in thread_data.o will be corrupted unexpectedly.
Specifically, if libaio specific parameter userspace_reap is given
on the command line, thread_data.o.description will be set to 1.
This fixes it by passing the correct pointer of ioengine specific
options to the command line parser.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
|
|
I got SIGABRT when I passed "bssplit=4k/10:64k/50:/40" to fio.
It should be "bssplit=4k/10:64k/50:32k/40" but I missed "32k" in
my typing. The bug turns out to be another typo just like mine:)
*** glibc detected *** /viki/lkml/fio/fio: free(): invalid pointer:
0x000000000071fd30 ***
======= Backtrace: =========
/lib/libc.so.6(+0x78bb6)[0x7ffff700cbb6]
/lib/libc.so.6(cfree+0x73)[0x7ffff7013483]
/viki/lkml/fio/fio[0x42c108]
/viki/lkml/fio/fio[0x42c475]
/viki/lkml/fio/fio[0x429194]
/viki/lkml/fio/fio[0x42a888]
/viki/lkml/fio/fio(parse_option+0x101)[0x42ac47]
/viki/lkml/fio/fio(fio_options_parse+0x95)[0x42e5b3]
/viki/lkml/fio/fio(parse_jobs_ini+0x3ae)[0x4103ae]
/viki/lkml/fio/fio(parse_options+0x112)[0x41198a]
/viki/lkml/fio/fio(main+0x3b)[0x460f5b]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7ffff6fb2c4d]
/viki/lkml/fio/fio[0x40a7c9]
Signed-off-by: Hong Zhiguo <zhiguohong@tencent.com>
Tested-by: Hong Zhiguo <zhiguohong@tencent.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Reported-by: Konstantin Ruvinsky <Konstantin.Ruvinsky@emc.com>
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Use the bold style with section number to mark syscalls.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
|
|
Right now we just segfault, if realloc() returns NULL. Handle
this a bit more gracefully, by just disabling logging if that
happens. Fio will also print an error for that case.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Many of fio's options are not described in the man page. Copy the short
text to the man page for those commands listed by:
./fio --cmdhelp=all | sed 's/ *:.*$//' | sed 's/ \+//' \
| while read line; do grep "\b$line\b" -q fio.1; if [[ $? -eq 1 ]]; then \
echo $line; fi; done
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Fix verifysort typo, broken formatting after roundrobin and tidy up some
grammar.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Problem:
When running fio with ioengine=sync, LAST_POS(f) state is not
reset after the file is closed. This causes workloads with
readwrite=randread and loops > 1 to fail verification if the
state of LAST_POS(f) at the beginning of the next loop is the
same as the io_u->offset. If that is the case, lseek in not
invoked, but the file position is at 0 and not at io_u->offset.
Proposed Solution:
Other ioengines, such as binject and fusion-aw, set f->engine_data
to 0 when closing the file. The sync ioengin uses f->engine_data to
store the LAST_POS(f) state. The proposed solution is to set
f->engine_data = 0 when closing a file in generic_close_file().
Signed-off-by: Juan Casse <jcasse@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Problem:
When specifying iodepth > 1 while doing random async io
without a random map (and using the default random generator), the
main while loop in do_io() will issue extra io in the amount of io
allowed to accumulate in the queue. For example, a job with size=24k
and bs=8k will perform 4 io instead of the expected 3.
Reason:
The above behavior occurs because the while loop in do_io()
will continue until the amount of "completed io" >= size, but io
in the queue are not accounted for because they have not been
completed by io_u_queued_complete().
Exceptions:
The above behavior does not manifest itself when:
using random_generator=lfsr bacause lfsr knows when to stop
generating, and get_io_u() inside the while loop returns NULL,
breaking out of the loop.
using random map because random map ensures that all blocks
are hit, and after they are all hit, no more are generated.
Proposed Solution:
Stop while loop based on bytes issued instead of bytes completed.
Signed-off-by: Juan Casse <jcasse@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We force local logging now, but that's not terribly useful
if you can't see the output from the backend. Make it follow
the usual rules of logging over the network if a backend, to
syslog, or locally.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Suresh reports:
-----
Does a requeued io_u work get submitted again through __get_io_u() ? If
so, the io_u item is taken out of requeue list & then the io_u->file is
set to NULL. We don't make a distinction about item being already setup
Vs a new io_u in __get_io_u(). In get_io_u(), when an io_u may come out
of requeue list, we depend on io_u->file to be non NULL to detect the
work as old pending one.
Am I missing something here or is this a bug?
1274 io_u = __get_io_u(td);
1275 if (!io_u) {
1276 dprint(FD_IO, "__get_io_u failed\n");
1277 return NULL;
1278 }
1279
1280 if (check_get_verify(td, io_u))
1281 goto out;
1282 if (check_get_trim(td, io_u))
1283 goto out;
1284
1285 /*
1286 * from a requeue, io_u already setup
1287 */
1288 if (io_u->file)
1289 goto out;
1290
"io_u.c" line 1286 of 1667 --77%-- col 3-10
1130 struct io_u *__get_io_u(struct thread_data *td)
1131 {
1132 struct io_u *io_u;
1133
1134 td_io_u_lock(td);
1135
1136 again:
1137 if (!io_u_rempty(&td->io_u_requeues))
1138 io_u = io_u_rpop(&td->io_u_requeues);
1139 else if (!io_u_qempty(&td->io_u_freelist))
1140 io_u = io_u_qpop(&td->io_u_freelist);
1141
1142 if (io_u) {
1143 io_u->buflen = 0;
1144 io_u->resid = 0;
1145 io_u->file = NULL;
1146 io_u->end_io = NULL;
1147 }
"io_u.c" line 1130 of 1667 --67%-- col 1
-----
We want a retry of a requeue to be identical, not some
new chunk (and drop the latter one on the floor). This got broken with
the introduction of the rings instead of lists, where a single if
section was split into two.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
David reports:
I'm using "fio --showcmd file.job" to convert a script to plain
commandline arguments and this fails if the directory specified in job
file does not exist. While this has to be an error if the job is being
executed, it should not be in context of --showcmd.
To reproduce:
$ cat job.fio
[global]
directory=/x
$ fio --showcmd job.fio
fio: /x is not a directory
fio: failed parsing directory=/x
fio: job global dropped
Expected output:
fio --directory=/x
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Only need to list LDFLAGS once on each line.
Signed-off-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
In general, make the existing document less ambiguous:
o Use consistent formatting/layout when describing options
o always refer to options with "--" prefix.
o reduce use of pronouns: "its" and "this" are often ambigous.
o remove use of "you" to describe user actions/choices
o add reference to examples/ directory
Signed-off-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
It's apparently a long on some platforms.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Fix a situation where we would spin prematurely and would hop over the
seed value.
Also, add some more comments to the code.
Signed-off-by: Alex Pyrgiotis <apyrgio@grnet.gr>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Jeff Moyer writes:
Our QE team noticed fio failures on recent kernels. I simplified the
job file to the following:
[global]
name=fio-mmap
rw=write
bs=4K
direct=1
end_fsync=1
verify=crc32
[file3]
size=100M
ioengine=mmap
mem=malloc
direct=1
After fio completes (and returns verify errors), the file is completely
full of zeroes.
Here is what the verify logic is doing:
static void do_verify(struct thread_data *td, uint64_t verify_bytes)
{
...
/*
* sync io first and invalidate cache, to make sure we really
* read from disk.
*/
for_each_file(td, f, i) {
if (!fio_file_open(f))
continue;
if (fio_io_sync(td, f))
break;
if (file_invalidate_cache(td, f)) <--------
break;
}
That invalidate cache call looks like so:
static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f,
unsigned long long off,
unsigned long long len)
{
...
/*
* FIXME: add blockdev flushing too
*/
if (f->mmap_ptr) {
ret = posix_madvise(f->mmap_ptr, f->mmap_sz, POSIX_MADV_DONTNEED);
(void) posix_madvise(f->mmap_ptr, f->mmap_sz, FIO_MADV_FREE); <-------
FIO_MADV_FREE can be defined as MADV_REMOVE, which will actually punch a
hole in the file (a hole the size of the entire file, btw). Of course,
unallocated blocks are returned as zeroes by the file system, so that
explains that!
---
For now, only do the MADV_FREE on non-files to avoid punching holes.
Further investigation is needed on the blockdev side, but at least
this should cure the immediate problem.
Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
