Skip to content

Instantly share code, notes, and snippets.

View nqn's full-sized avatar

Niklas Nielsen nqn

  • San Francisco, CA
View GitHub Profile
layout
documentation

Getting Started

Downloading Mesos

There are different ways you can get Mesos:

[15:10:30] <niq__> 1) How is the API supposed to scale? Getting additional values requires sending more lambdas to update? How is the corrections API going to look like? A callback to add correction messages to the queue?
[15:10:36] <niq__> 2) Safety in terms of hogging the thread from a module. Even in the Isolator modules (as they are today), the implementer is required to make those calls async. If he/she want to do nasty things - they can do that anyway
[15:11:43] <niq__> 3) I then wanted to understand; did you propose the 'push' to avoid getting the value of an already fired future? (the busy loop previous referred to)? That's why I suggested making it a queue initially
[15:11:53] <niq__> Did you get 1, 2 and 3?
[15:12:03] <jieyu> Yes
[15:12:27] <jieyu> regarding (1), getting additional values
[15:12:40] <jieyu> are you refering to getting say, slave's total resources
[15:14:44] <jieyu> u there?
[15:14:56] <jmlvanre> he'll be back in a second
[15:15:01] <jieyu> k
@nqn
nqn / hook.md
Last active August 29, 2015 14:19

Hook

Similar to Apache Webserver Modules, hooks allows module writers to tie into internal components which may not be suitable to be abstracted entirely behind modules but rather let's them define actions on so-called hooks.

The available hooks API is defined in mesos/hook.hpp and for each hook defines the insertion point and available context. An example of this context is the task information which is passed to masterLaunchTasksHooks. Some hooks have return values which allow hooks to hang off metadata of the object in flight.

Cherry pick In progress In review Landed in OSS Owner Shepherd
Label undecorator (30961, 30962, 31016, 31028, 31017) X Niklas Adam B
Added concurrency protection within the SASL auxprop plugin code (31960) X Till Adam B
Split cram_md5 authenticator into declaration and implementation without functional changes (31961) X Till Adam B
Simplified master authentication (31957) X Till Adam B
Revised authenticator interface to allow for two fold implementations (27760) X Till Adam B
FramworkInfo.id fix X Kapil Adam B
Disable libprocess HTTP endpoints X Alex R Adam B
Apache Mesos 0.22.0 Released
The latest Mesos release, 0.22.0 is now available for download. This release includes several major features:
Support for explicitly sending status updates acknowledgements from schedulers; refer to the upgrades document for upgrading schedulers.
Rate limiting slave removal, to safeguard against unforeseen bugs leading to widespread slave removal.
Disk quota isolation in Mesos containerizer; refer to the containerization documentation to enable disk quota monitoring and enforcement.
Support for module hooks in task launch sequence. Refer to the modules documentation for more information.
Anonymous modules: a new kind of module that does not receive any callbacks but coexists with its parent process.
New service discovery info in task info allows framework users to specify discoverability of tasks for external service discovery systems. Refer to the framework development guide for more information.
Futher more, many bugs have been fixed in this release. For full release notes with
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0317 22:47:00.189301 15914 process.cpp:1681] libprocess is initialized on 172.16.0.14:39932 for 40 cpus
I0317 22:47:00.189759 15914 logging.cpp:177] Logging to STDERR
I0317 22:47:00.190650 15914 exec.cpp:132] Version: 0.21.1
I0317 22:47:00.191193 15997 exec.cpp:182] Executor started at: executor(1)@172.16.0.14:39932 with pid 15914
I0317 22:47:00.202662 15975 exec.cpp:206] Executor registered on slave 20150225-134652-201330860-5050-28873-S9
I0317 22:47:00.203483 15975 exec.cpp:218] Executor::registered took 82347ns
I0317 22:47:00.218021 15965 exec.cpp:293] Executor asked to run task 'mom_d4462b3c-3fac-494a-4380-d7764ab9301d_slave.610e41b8-ccf7-11e4-8c79-5eb27fd3ddfa'
I0317 22:47:00.218127 15965 exec.cpp:302] Executor::launchTask took 64049ns
I0317 22:47:00.221932 15972 exec.cpp:525] Executor sending status update TASK_RUNNING (UUID: 28fa9309-ffc0-4f32-8d95-30135227ac97) for task mom_d4462b3c-3fac-494a-4380-d7764ab9301d_slave.610e41b8-ccf7-11e4-8
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0317 21:26:04.632277 1397 process.cpp:1681] libprocess is initialized on 172.16.0.17:42923 for 40 cpus
I0317 21:26:04.632593 1397 logging.cpp:177] Logging to STDERR
I0317 21:26:04.633370 1397 exec.cpp:132] Version: 0.21.1
I0317 21:26:04.633939 1703 exec.cpp:182] Executor started at: executor(1)@172.16.0.17:42923 with pid 1397
I0317 21:26:04.785586 1724 exec.cpp:206] Executor registered on slave 20150226-232548-251662508-5050-4101-S2
I0317 21:26:04.786417 1724 exec.cpp:218] Executor::registered took 100484ns
I0317 21:26:04.788089 1749 exec.cpp:293] Executor asked to run task 'mom_c81c2c85-cfde-4083-54c9-c168b32250d3_slave.3735107b-ccec-11e4-8c79-5eb27fd3ddfa'
I0317 21:26:04.788179 1749 exec.cpp:302] Executor::launchTask took 59773ns
I0317 21:26:04.791371 1694 exec.cpp:525] Executor sending status update TASK_RUNNING (UUID: 4a4a59c1-c594-4f0e-8625-f1706f1ef91b) for task mom_c81c2c85-cfde-4083-54c9-c168b32250d3_slave.3735107b-ccec-11e4-8c
Log file created at: 2015/03/17 01:39:55
Running on machine: srv3.hw.ca1.mesosphere.com
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0317 01:39:55.909622 6 logging.cpp:172] INFO level logging started!
I0317 01:39:55.909806 6 main.cpp:156] Build: 2015-03-16 23:09:06 by
I0317 01:39:55.909821 6 main.cpp:158] Version: 0.22.0
I0317 01:39:55.909827 6 main.cpp:165] Git SHA: a2f90f20f421fca5f9ea866ade34a220dd00fc57
I0317 01:39:55.909898 6 containerizer.cpp:110] Using isolation: posix/cpu,posix/mem
I0317 01:39:55.910226 6 main.cpp:200] Starting Mesos slave
I0317 01:39:55.910872 22 slave.cpp:174] Slave started on 1)@172.16.0.13:31114
diff --git a/src/Makefile.am b/src/Makefile.am
index 3059818231c46484039d179cd6916932eff6cd68..e1efb3138159c31f28245a2ed09f5b8d7df941d1 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -74,6 +74,13 @@ BUILT_SOURCES =
CLEANFILES =
EXTRA_DIST =
PHONY_TARGETS =
+LDADD =
+
#!/usr/bin/env python
import argparse
import os
import subprocess
import sys
import tempfile
import time
class Process:
def __init__(self, args):