Skip to content

Instantly share code, notes, and snippets.

@kwilczynski
Last active July 12, 2022 16:58
Show Gist options
  • Save kwilczynski/66fe48a1339454798af4152dd9c34a6e to your computer and use it in GitHub Desktop.
Save kwilczynski/66fe48a1339454798af4152dd9c34a6e to your computer and use it in GitHub Desktop.
Assorted interview questions (across networking, Linux, OS internals, AWS, Docker, etc.)

Generic questions

  • How does a CDN (Content Delivery Network) work?

    • What is a "POP"?
    • What are the benefits of a CDN, and what these tend to offer (briefly)?
    • What is "geo-IP", what is it used for?
    • Can CDN be used for load-balancing?
  • Can you describe how DNS works?

    • Ask about TCP vs UDP, what problems is TCP trying to solve?
      • What is "XFER" (zone transfer)? What potential problems allowing this query type can introduce?
    • What is "SOA"? What does it include?
    • Ask about DNS-over-TLS.
  • What is the difference between /32 and 255.255.255.255, what notation is used in each case?

    • Ask about "classful" vs "CIDR" vs "VLSM" notation?
    • Is FLSM the same as VLSM?
  • What is the difference between unicast, broadcast and multicast?

    • How do TCP and UDP work in each case?
  • What is the difference between /32, /22, 255.255.255.255.0, /31, /0, 255.255.255.255.252, /24 and 255.255.255.255; and which is one would not be correct?

    • What does 0/0 mean?
  • What is the difference between ifconfig and ip (from iproute2)?

    • How to assign another IP address to a network interface? (secondary IP vs an alias)
  • How many usable IP addresses are contained within networks /30, /29 and /24 and /22?

    • Ask about how many bits and bytes IPv4, IPv6 and ARP address has?
    • Does IPv6 include IPv4 as its subset?
    • What is "IPv4-mapped IPv6 address" (IPv4 address embedding)?
  • Can you describe the difference between TCP and UDP?

    • Ask about different states in the TCP state machine e.g., TIME_WAIT, TIME_WAIT2, etc.
      • What problems can this cause, and how to fix this? Ask about close() vs shutdown() on a socket.
        • Follow-up asking about what it means for a TCP connection to be "half-open" or "half-closed" (ask about SHUT_WR flag of shutdown(), etc.)
  • UDP vs TCP - why would you use UDP for anycast DNS?

    • Ask about "anycast" vs "unicast" vs "multicast", what is the difference?
    • Does multicast (and broadcast) works with TCP?
      • If not, then why not?
  • Does UDP use ICMP for anything?

    • What might it use for, if at all?
    • Why blocking ICMP might be a bad idea? Would you block it? if so, then why?
  • Can you load-balance UDP?

    • When would it make and when would make sense and when it would not?
  • You send a ping (ICMP) or packet (TCP/IP, with just SYN flag set) to a server that does not respond or does not exist at the moment, what stops it (the ICMP or IP) from being transferred by connected systems forever?

    • Ask about common flags (bits) set in IP and TCP headers, etc.
  • What is the lowest TCP port number?

    • Ask whether it is 0 or 1 and if 0 then why not 0?
    • Ask about bind() on port 0? Is it possible? What will happen?
    • Ask about the privileged range 0 - 1023 (or < 1024), why is it special?
      • Ask about CAP_NET_BIND_SERVICE Linux capability (setcap, etc.).
  • What is TCP SACK?

    • How does it work?
    • What might be side effects if you disable support for SACK on Linux?
  • The TCP frame has a URG pointer field, when is it used?

    • Ask about PSH flag?
    • Follow-up and ask about other possible flags?
  • What does the PSH flag do?

    • Ask about URG flag?
  • Can the RST packet have a payload?

  • Can an SYN packet have a payload?

  • Can an SYN+ACK packet have a payload?

    • Ask about "TFO" (TCP Fast Open)?
  • What does the IP_FREEBIND socket option do?

    • Why is it useful?
      • Follow-up asking why this is useful for software integrating with systemd (allowing to bind to addresses not owned by any interface).
      • How is this being set? (ask about setsockopt())
    • Ask about any other useful options e.g., IP_TOS, IP_MTU, etc.
    • Ask whether is it the same as SO_REUSEADDR flag?
  • What is a stupid (or silly) window syndrome?

    • Ask about TCP flow control and window scaling?
  • When using git and resolving a merge conflict, what is the difference and meaning of "theirs" and "ours"?

  • What is a difference between a Hash (Hash Table) and an Associative Array? Could you implement a dictionary/hashtable from scratch?

    • Ask about asymptotic complexity (big-O), and why it is not quite O(1)?
    • Ask about different types of implementations.
    • How to resolve a collision? (two values map to the same entry)
  • What asymptotic complexity (the BIG-O notation) quicksort (qsort) has?

    • Can you remember which one would merge sort have?
      • Quick Sort: O(n*log(n)) for average and best, worst is O(n^2).
      • Merge Sort: O(n*log(n)) across the board.
    • When would you use merge sort over quick sort?
      • Quick Sort: constant space.
      • Merge Sort: space depends on what you are sorting (e.g., an array is O(n)).
    • Which algorithm do you think is faster? What about space efficiency?
  • Can you talk about Factory and/or Singleton design patterns? Where does Singleton become useful?

  • Explain the difference between strong vs weak typing. Are you familiar with terms dynamic vs static/duck typing?

    • Ask what a “duck typing” is and where have you seen such principle applied?
  • What is "black box" testing? How does it differ from "white box" testing?

  • Given a string "catsatonthemat" how would you split it into separate words?

    • Recursive or linear approach? Ask about strtok(), etc.
  • Can you teach someone about common HTTP headers?

    • What is the significance of "Host" header and TLS?
      • Ask about CN and SNI.
  • What's the 403 HTTP response status code?

  • Can you talk about Factory and/or Singleton design pattern?

  • Can you describe the difference between Functional and/or Object-Oriented Programming?

  • What's the upside/downside of Micro-Service (Service Oriented; SOA) Architecture and Monolithic Architecture?

  • What is the main reason people are moving toward Micro-Service Architecture?

  • Can you describe what a RESTful API is about? How it should be designed?

    • Do you know what GraphQL is?
  • Explain the difference between “strong” and “eventual consistency”?

  • Have you heard about CAP theorem? If so, then what is it?

    • Ask what does consistency, availability and partition tolerance mean?
      • Can a (or any) system achieve all three?
    • Ask about any real-life application of this theorem?
  • Kafka: From the CAP theorem, where does Kafka aims to be?

    • Ask about why CA and not AP?
  • What's the upside and downside of microservices (or service-oriented; SOA) architecture and monolithic architecture?

    • Ask about the main reason people are moving towards microservices architecture?
  • What's a container? How does it work? How is it different from a traditional VM?

  • The difference between RAID-0, RAID-1 and RAID-5. What's the advantage and disadvantage? Which one to use for securing fault-tolerance?

  • What is "Page Cache" and "Memory Page"? Can you explain what a "Page Fault" is?

    • Ask about the type of memory faults (minor vs major).
    • Ask about swap space, and what happens when you disable swap on the system under pressure?
    • Ask why on modern 64-bit architecture VSZ (virtual address) space is often much larger (reported) than even available RSS (resident set size) space? How is that even possible?
      • Ask about the sparse memory page (Zero Page).
    • What is a "context switch"?
    • What are Huge Pages?
      • Why there are problems here?
      • Different page sizes 4KB vs 2MB vs 1GB, what are the issues with these? I/O performance?
    • What do kswapd (kswapd#), pdflush and bdflush (buffer-dirty-flush) does?
      • Ask about the free_pages_high watermark.
  • What is the difference between "symmetric" and "asymmetric" encryption?

    • Ask about performance differences, which one is faster?
  • How does TLS work?

    • Ask about encryption vs authentication.
      • Does authentication provide encryption?
      • When talking about encryption, ask about expired TLS certificate, why does it work?
      • What is the difference between "symmetric" and "asymmetric" encryption?
      • What parts of TLS are asymmetric and what parts are symmetric encryption?
    • What is a SAN certificate?
      • Ask about SNI and CA.
  • Can you explain the difference between “authentication” and “authorization”?

    • Does one imply the other?
  • What is "CQRS"? Why is it a common pattern? Is it useful?

    • Ask about other patterns.

Open-ended questions (the more the detail and depth of knowledge the better):

  • How does a typical PC (assuming 8086 architecture) starts (boots up)?

    • Ask about "Real Mode" vs "Unreal Mode" vs "Protected Mode".
      • Ask about memory layout in each. What is "segmentation"?
    • Ask about grub stage 1, 1.5, and 2.
    • Ask about EFI vs old-style BIOS.
  • Why is timekeeping important? What issues are related to time drifting?

    • What stores time on a PC or a server, etc?
    • Ask about ntpd and how it works.
    • Ask about jiffies and NOHZ (tickless) Linux kernel.
  • Describe with as many details as possible what happens after you type "http://www.google.com" in the browser (or curl, or whatnot) and press enter.

  • What is the difference between su and sudo?

    • What does the following do: su -
    • What does sudo su then do?
    • sudo -i vs sudo -s - what is the difference?
  • What is the result of the following typed in Bash shell: mkdir a; cd a; touch b; ls > c; what is "c" going to contain and why?

    • Bash parsing order?
    • Describe library (or system) calls involved?
    • How would you find out?
    • How do the strace and/or ltrace commands work?
    • noclobber and > vs >|
    • Difference between &>xyz and >&xyz?
  • What happens when one runs "cat file.txt" in a terminal window displaying an SSH session on a remote Linux machine. What is happening on both ends of the connection, on both of these the machines?

  • Describe what happens when you type the following in Bash: ls -l *.txt

    • Bash parsing order?
    • Describe library (or system) calls involved?
    • How would you find out?
    • How do the strace and/or ltrace commands work?
  • How to check for a command availability in the Bash shell?

    • Ask what does PATH actually is for?
    • Ask about which vs command -v.
    • Ask about Bash built-ins and aliases
    • Ask about $ ls vs $ \ls, what does it do?
  • If you add a directory to $PATH environment variable, should you add in front, or at the end of the variable? Most importantly, why?

    • Ask about how PATH is evaluated? How are the entries read?
    • Ask about cache and the hash built-in.
  • Given the following line typed in Bash, explain what happens when you sent ^C to it? A) ls -l *.txt | grep 'something'; B) ls -l *.txt | head -n 10

    • Describe how commands are executed now, what would be the process tree?
    • Describe library (or system) calls involved?
    • How would you find out?
    • How do the strace and/or ltrace commands work?
    • How signals are working in this case?
  • What you do when you see "mv * /tmp" error with "bash /bin/mv: Argument list too long"?

    • What if there is file starting with "-" (dash/hyphen) in its name?
  • On Linux, what process, except the init itself, is not a child of init process?

    • Ask about what is a thread? What is a kernel thread more precisely?
    • Ask about the square-brackets entries in the ps -ef / ps aux output?
  • On Linux, can a thread have its own separate/private file descriptor (e.g., TCP socket, open file, etc.) that is not shared with other threads? If so, how? If not, why?

    • Ask about unshare() system call?
      • Follow-up asking about containers on Linux (e.g., Docker containers).
    • Ask about the CLONE_FILES flag for the clone() system call?
      • Follow-up asking about the FD_CLOEXEC flag used with fcntl().
  • How to kill a defunct (zombie) process?

    • Do child processes exit when parent exists? (Ask about prctl)?
      • Follow-up with a question about PR_SET_CHILD_SUBREAPER for prctl.
    • Role of the init system?
    • The SIGCHLD signal? Difference between SIGINT, SIGTERM and SIGKILL?
      • Follow-up with a question about SA_NOCLDWAIT.
    • How to check if PID exists? (Ask about kill -0 <PID>).
    • Controlling terminal, process group, session leader, etc.
    • Ask about kill -0, kill -<PID> and kill 0
    • Ask about signal propagation inside a Docker container.
  • Write a program that exits with a status code of 256 (integer).

    • Ask if this is even possible?
      • Follow-up with a question about process status (8 bit; lowermost) and Linux POSIX signals.
    • Ask about wait()/waitpid() from GNU libc (glibc)?
      • Follow-up about waitid() and possibly the "si_status" field from the sigaction struct (siginfo_t), etc.
    • Ask about exit()?
      • Follow-up asking what is the range of possible exit codes a process can exit() with?
  • What nohup do, why do we use it?

    • Ask about process groups and controlling terminal (TTY, etc.).
    • What does SIGHUP do?
  • What is LD_PREOLOAD for?

    • Ask about setuid/setgid binaries?
    • What is function interposition (or simply symbols)?
    • Ask about rpath in ELF binaries and how loader (and liner works).
    • How does it differ from LD_LIBRARY_PATH?
  • How does the ptrace and/or ltrace commands work?

    • Ask about issues running this inside a Docker container.
  • Explain the difference between "date > file" and "date | tee file" (typed in the Bash shell).

  • How to terminate the connection by closing a socket without killing or restarting a running process?

    • Ask about /proc/<PID>/fd.
    • Ask if everyone can read /proc/<PID>/fd, and follow with why not?
    • Ask about lsof and "deleted", what these are? Ask about unlink() in relation to close(), etc.
    • Ask about /proc, what is it?
      • Why reading from files in /proc work, but things like select/poll/epoll might fail?
      • Can you set an event listener on a file in /proc (or sysfs?)
  • How fork(2) works? What return value is sent to parent and child?

    • What happens when the fork() is invoked?
    • What is s syscall (system call)?
    • Difference between fork() and vfork()? What is Cow (Copy on Write) in this case?
    • Does SIGKILL, SIGSTOP and SIGCONT work on zombies?
  • Explain what impact they might have and why: scheduling policy and priority, CPU affinity, I/O scheduling class and priority, network shaping (qdisc / packet scheduler) and NUMA.

    • Ask about tools e.g., nice, ionice, chrt, taskset and numactl.
      • Ask which system capability is needed by some of these tools, which CAP_SYS_* would it be?
    • Ask about ticks e.g., 250 Hz, 1000 Hz, NOHZ (tickless) and potential role and/or impact here?
  • How the /bin/ls command works, and how to implement it?

    • Ask about Unix permission model and permission bits.
    • Ask about umask (what is the difference between 022 and 027).
    • Ask about openat, getdents and readdir, stat, ctime, mtime, atime, crtime, etc.
  • What is VFS?

    • Ask about file systems in general.
    • Ask about file (struct), path (struct), i-node(struct), stat, dentry (struct), superblock, metadata, etc.
    • Ask why superblock - why it was located in multiple places and why it mattered (perhaps they remember ATA/SCSI spindles).
    • Ask about FUSE?
  • What is sync command and what system call does it use?

    • Ask about side effects of using sync system call?
    • Can synchronisation be triggered only for a specific file (or a file descriptor)?
      • Follow-up asking about fsync().
    • Are operations of file such as rename() safe without using sync()/fsync()?
      • Follow-up asking about whether rename() is an atomic operation or not?
        • Ask about file systems such as ext4 journaling and data=ordered vs data=writeback?
  • Let's assume that I don't know much about how computer networks work, how would you explain to me how the Internet works?

  • Are you familiar with different memory allocators?

    • Ask about ptmalloc2 vs tcmalloc vs hoard vs jemalloc.
    • Can memory be fragmented on Linux? Ask about Virtual Memory (VMS) vs Real Memory (RSS) and Memory Pages (buddy algorithm).
  • How does Bash shell execute shell scripts given the #! (shebang), etc. What is involved and how does it work?

  • How would you explain to someone who is not necessarily technical, what is a Cloud Computing, and how it works?

  • Scenario: A young and inexperienced System Administrator has done the following: "chmod -x /bin/chmod"; onto a very important server, and now you have been tasked with fixing the problem, how would you do it?

    • Ask about install -m 755.
    • Ask about using scripting programming language, etc.
    • Ask why copying over different binary works, and why? (check for i-node, etc.).
  • Scenario: Imagine, that "/dev/null" is gone/missing, how would you re-create it (recover)? Is it even possible to remove "/dev/null"?

    • Ask about udev vs static /dev.
      • Ask if knows about devtmpfs file system?
      • Ask about /proc, what is it?
      • Why reading from files in /proc work, but things like select/poll/epoll` might fail?
      • Can you set an event listener on a file in /proc (or sysfs?)
  • Scenario: You have a directory with a VERY large number of files, and you need to remove all of them quickly, but you cannot remove a top-level directory, and you are finding that "rm -Rf *" does not work (an error shows up due to a large number of arguments), what do you do?

    • Ask about the usage of "find" with "-exec" or "parallel", or maybe "xargs"?
    • Ask about rsync (with the -a option), and how this might be implemented?
    • What if you have sub-directories you do not want to remove?
  • Scenario: How to write an Internet Archive crawling bot, given that you have 10,000 worker nodes.

    • CPU and memory are limited, but disk storage is not.
    • How to co-ordinate?
  • Scenario: You alerting system is suddenly throwing a lot of alerts (about memory, unable to fork, etc.) and notifications from a very important server. This host runs a curial business application and this machine at the moment is still processing a very important job (irrelevant what it is), thus you want to sing-in on the server and inspect what is going on, but you are greeted with the following message: "bash: fork: retry: Resource temporarily unavailable bash: fork: retry: No child processes". You then realise that you cannot connect to the machine at the moment due to this error, but... thankfully you still have one single SSH session opened with Bash shell prompt on this machi from before few hours when you were doing some maintenance on the server. Now, you are in and you have root shell, what do you do? And what might be the reason for the error you have been seeing now?

  • Docker: What is the difference between containers vs virtual machine?

    • Ask about hardware emulation, hardware partitioning vs hypervisors and paravirtualisation?
  • Docker: What is the difference between CMD and ENTRYPOINT in a Dockerfile?

    • What is the difference between CMD "date" and CMD ["date"]?
    • What can you do with a script placed in ENTRYPOINT? Some common use cases?
  • Docker: What you might need to have a "fake init" (or simply "init") process inside a container? What problem such init process is trying to solve?

    • What signals Docker sends when you stop a container?
    • How are signals propagated inside the container?
  • Docker: Imagine that there is no Docker, how would you create a "container" manually?

    • Ask about UTS namespaces (PID namespaces, user namespaces, etc.).
    • Ask about cgroups? What are these for? Can you have a "container" without a cgroup?
    • Ask about bind mount, and why it might be relevant?
    • Ask about Linux capabilities, do you know about CAP_SYS_ADMIN?
      • Bonus: Can you create, bind, listen on a socket on port 80 as a non-privileged user?
    • Do you know about unshare? What is a process execution context?
    • Do you know about setns and/or nsenter?
    • Difference between chroot vs pivot_root vs switch_root in terms of a container?
      • Ask about a container file system in terms of initrd and initramfs, how do these differ? could you use these?
  • Docker: You created a Docker image with an application written in Go but it is too heavy. How to reduce the size of the image? What are multi-stage builds?

  • AWS: How does the SSH key-pair is deployed on an EC2 instance, so that you can connect to it with the right user?

  • AWS: Can you explain the difference between NACL (Network ACL) and Security Group in AWS?

    • When and where would you use one or the other?
    • Do you know of any limitations of NACLs and Security Groups?
  • AWS: What are the most common challenges when running instances and services in an Auto-Scaling Group?

  • AWS: What is a difference between a NAT instance and NAT Gateway (the AWS managed service)?

    • How would you make NAT instance highly available and resilient, if you cannot use NAT Gateway?
  • AWS: What is the difference between "gp2" and "io1" EBS volume type?

    • What is an EBS volume?
    • What happens with gp2 volume when you burst/cause a surge in IO operations? What are the possible side effects?
  • AWS: What is a "cold startup" problem when using Lambda functions, and how to mitigate it?

  • AWS: Scenario: You are load-testing your application to assess the baseline. The application is deployed in AWS on a number of EC2 instances. Your test and/or benchmark shows problems with erratic latency and relatively slow response times and the low number of requests per second (RPS) to the one you expected. What might be happening and what can you do to ensure that your load-testing yields sensible results?

    • What if we replace EC2 instances with ones in an Auto-Scaling group?
  • AWS: Scenario: Given that Amazon AWS does have sometimes a very high profile outages (for example the infamous EBS outage, or recent S3 problems, etc.), what would you do to build an infrastructure and/or a service to operate for and mitigate such large scale issues? What would be your approach?

    • How would you find out that there is a large scale outage ongoing in AWS?
  • AWS: Scenario: You are dealing with an infrastructure that has been migrated to Amazon AWS from a physical (bare metal) date centre (the migration was simple "lift and shift" style, nothing has been engineered specifically for AWS), and now your biggest concern is not necessarily performance, but the costs of operating this infrastructure in which is now operating wholly from AWS. What would you change and where to try to bring the costs down?

    • How to identify where the money is being spent at the moment?
    • How to monitor current running costs?
  • Kubernetes: Which Master (Control Plane) processes run on all of the Master nodes?

    • What processes (binaries) constitute of Master and Worker nodes?
    • Ask about kube-controller-manager and kube-scheduler.
  • Kubernetes: What is a PVC?

    • What is PVC used for and why?
    • What problems does PVC introduce?
      • Follow-up with asking whether you can resize a PVC-based volume?
  • Kubernetes: Scenario: You were given a container that contains a legacy application and were tasked to make it work (deploy it to production, if you wish) in the production environment. The container ships a legacy application crated by some software vendor which you have very limited knowledge about (there is some generic documentation, but there isn't any source code available, etc.). This legacy application expects another service (it's runtime dependency, for example, a data store) to be accessible via localhost on a port 31337, and you cannot change thi address (location) as it has been hardcoded into the third-party application and there is no option to change it in a configuration file o otherwise. How would you make this work? How would you make this application that requires something on localhost work within a Kubernetes cluster and make it connect to the desired service operating somewhere else (a different set of Pods, etc.)?

    • Ask about Linux namespace (the network one)?
      • Should Docker be used as the runtime (often the default), then how does it put all the containers in the same namespace? (the "pause" container, the "infra" container, etc.)
  • Kubernetes: What is the difference between request and limit for resources allocation?

    • Ask about burst vs guaranteed Pods, what does each mean? Ask about memory over-provisioning?
    • Ask about limiting I/O? Even possible?
    • Ask about what it teams that a CPU is throttled?
  • Kubernetes: What is the difference between readiness and liveness probe?

  • Kubernetes: Challenges around persistent volumes?

  • Go: What is interface embedding?

  • Go: How would you do a mock in testing?

  • Go: What happens when you read from a closed channel?

    • How is this feature useful?
    • Can you write to a closed channel?
  • Go: You are trying to write to or read from a nil channel, what are the challenges with such an operation?

  • Go: What happens when you read from an empty channel? How to add read timeout or implement non-blocking read?

  • Go: Can a string be nil?

  • Go: What is the main property of a string in Go?

  • Go: When using range to iterate over a map in Go, are the keys always in the same order? In what order are the keys of a map in Go when using range?

  • Go: What would you do if you need a map displayed in a fixed order?

  • Go: What is the possible side effects when you convert from a string to a byte slice?

  • Go: What is the difference between len and cap for a slice and a map?

  • Go: Can you use an equality operator (the “==”) in Go to compare two structs? If so, then how does it work?

  • Go: You have a simple map that you use only for looking up keys (e.g., a membership check) and you’ve noticed that the values are of a struct{}{} type, what is it? Is this even valid? Would a simple bool type be better here? If so, then why? If not, then why not? How much space does struct{} occupies?

  • Go: What is the difference between “var n []int” and “n := []int{}”?

  • Go: You have struct with multiple fields, and decided to serialize it into JSON, XML, gob, etc., which fields would be included and which wouldn't be in the results?

  • Go: What is the difference between an interface{} type and an actual interface in Go. Is the interface{} type a pointer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment