Q - [kju:] Archives - MikesMassiveMess

As I'm currently rewriting Q - [kju:] from scratch, I have to tackle a longstanding QEMU frontend problem: vga buffer of the guest os in one application - vga output window in a viewing/controlling app.
The most economic would be a buffer that could be accessed by both apps. Shared Memory.
There was once a shmem patch submitted to the QEMU dev list, but it got dropped in favor for a vnc solution. As I'm not satisfied with vnc speed, I took a stab at the shmem patch.

shmem on OS X, it's a no-go...
Shmem would be the most geleagnt way of creating shared memory. Unfortunateley, Apple has limited shmem to a lousy 4MB - for the whole system!Blade:~ mike$ cat /etc/rc | grep shmmax sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024Manipulating rc is not my thing. Setting shmem with 'sysctl -w kern.sysv.shmmax=10485760' to 10MB on the fly needs to much privileges.

Distributed Object, anybody?
I'm already using DO to communicate between the controoling ap and the QEMU instances. The problem with DO is, that all the messages are copied and sent to the corresponding object. So accessing a buffer in a distandt object with - (void *) screenBuffer; will copy and send the whole buffer, not just a memorypointer. The advantage of the DO, you could have the viewer on a different machine - (Q - [kju:] Server anybody? :)).
So I wrote a methode, that just sends updated segments of the screenBuffer.
Here is a downstripped version of the Distributed Object (DO) I used to transfer changes in the guests videoram to the viewer:

DO in data reciever process:

@protocol RecieverObjectProto
- (void) updateBufferWithData:(NSData*)data
            start:(size_t)start
            length:(size_t)length;
@end
 
@interface RecieverObject : NSObject <RecieverObjectProto> {
    void *destinationbuffer;
}
- (void) updateBufferWithData:(NSData*)data
            start:(size_t)start
            length:(size_t)length;
@end
 
@implementation RecieverObject
- (id) initWithSender:(id)sender
{
    self = [super init];
    if (self) {
 
        // our buffer
        destinationbuffer = malloc(1024*1024*10);
 
        // Open a connection, so the client can connect to us
        NSConnection *connection = [NSConnection defaultConnection];
        [connection setRootObject:self];
        if ([connection registerName:@"buffer"] == NO) {
            NSLog(@"Could not establisch buffer server");
            return nil;
        }
    }
    return self;
}
 
- (void) dealloc
{
    if (destinationbuffer)
        free(destinationbuffer);
    [super dealloc];
}
 
- (void) updateBufferWithData:(NSData*)data
            start:(size_t)start
            length:(size_t)length
{
    UInt8 *pixelPointer;
    pixelPointer = destinationbuffer;
    memcpy(&pixelPointer[start], [data bytes], length);
}
@end

DO in data creator process:

@interface SenderObject : NSObject {
    RecieverObject* qDocument
    void *sourcebuffer;
}
- (void) sendBufferUpdate;
@end
 
@implementation SenderObject
- (id) init
{
    self = [super init];
    if (self) {
        // connect to the QDocument object
        qDocument = [[NSConnection
            rootProxyForConnectionWithRegisteredName:@"buffer"
            host:nil] retain];
        if(!qDocument) {
            NSLog(@"Could not connect to buffer");
            return FALSE;
        }
 
        sourcebuffer = malloc(1024*1024*10);
 
        return self;
    }
    return nil;
}
 
- (void) dealloc
{
    if (sourcebuffer)
        free(sourcebuffer);
    [super dealloc];
}
 
- (void) sendBufferUpdate
{
    size_t start = 1024; // start to send data from offset 'start'
    size_t length = 1024; // send 'length' bytes
    UInt8 *pixelPointer;
    pixelPointer = sourcebuffer;
 
     // note: I use dataWithBytesNoCopy => one time less copying
    data = [NSData dataWithBytesNoCopy:&pixelPointer[start]
                length:length freeWhenDone:NO];
 
    // lets send the data
    [qDocument updateBufferWithData:data start:start length:length];
}
@end

Downside of the shared Object: data is copied at least 2 times: when the message is created and when the data is writen from the message to the reciveers buffer. Faster than vnc... but data objects are sometimes not released as fast as created => incredible memory footprint.

finally, mmap!
With mmap you can load a file into memory. If more than one processes mmap the same file, all processes can access the same mmap, that way we have shared memory, and a convenient way to share the adress: the file.

First we need a file the size we need shared memory...

// we need a dummy file, this gives us the possibility to
// access the shared memory over the filesystem
int bufferSize;
int fd, ret;
bufferSize = 1024*1024*10;
void *dummyFile;
dummyFile = malloc(bufferSize);
 
// open trunkate/create read/write for everybody
fd = open("/tmp/myRamFile", O_CREAT|O_RDWR, 0666);
if (fd == -1) {
    NSLog(@"Could not open '/tmp/myRamFile'");
}
ret = write(fd, dummyFile, bufferSize); // create the file
ret = close(fd);
free(dummyFile);

...after we attach, from multiple processes.

// open the file with read & write access
fd = open("/tmp/myRamFile", O_RDWR);
if(!fd)
    NSLog(@"Could not open '/tmp/myRamFile'");
 
// load the file into memory, shared, read & write access
int buffersize = 1024*1024*10;
void *sourcebuffer = mmap(
    0, buffersize, PROT_READ|PROT_WRITE, MAP_FILE|MAP_SHARED, fd, 0);
if(!sourcebuffer)
    NSLog(@"Could not mmap '/tmp/%s.vga'");
 
// once the file is mapped, we can dispose of the filehandle
close(fd);
 
//... do something intresting with the shared memory, then unmap
if (sourcebuffer)
    munmap(sourcebuffer, buffersize);

Downside? Well if more than one process make changes to the mmaped file, it's writen from time to time (normally when the last process exits) but I can live with that.
I hope to comit the rewrite of Q - [kju:] soon 🙂

Mike

Category: Q – [kju:]

Updates & Twitter

Q-0.9.1d104

shared memory on Mac OS X

Q-0.9.0

Q-0.9.0d86