Dark Developments Where Knowledge Meets Power

8Nov/120

Artificial Intelligence

Posted by Ollie

As all our forum members who read the introductions thread know, I am currently studying Artificial Intelligence (AI) at university... what some of you might not know is what it is. (If you are not one of those people you may find this article a waist of time to read, or still of interest... If the former please wait until I publish something more interesting.)

What is machine intelligence?
So when I tell people I study AI the number one response I have gotten is "is that aliens?" and the answer is NO. For this reason I've decided first to define it.

Artificial intelligence is the field devoted to making "computers" (a computer can be any digital system) learn. It seems intuitive however a lot of people seem to not quite make the link in their mind. How some people think artificial means "aliens" I am not sure, as artificial by definition means, man made... not of nature.. Aliens are not man made!

What kind of things do you do?
Well here I should make a distinction between Artificial Intelligence and Machine Intelligence as they are often both bundled together and I may well end up posting about both

  • Artificial Intelligence: This is the study of replicating the cognitive process via digital means to create a system that learns - Mainly when I post about this it will be in relation to Neural Networks which are a collection of programmed "brain cells" (neurons).
  • Machine Intelligence: This is the study of creating intelligent robotic systems, but not necessarily using artificial means (a lot of the time artificial means are used though). An example of an intelligent machine that does not use artificial means of intelligence is a biologically intelligent robot. In the University of Reading, we use rat brains to control robots. All the inputs and outputs given to / from these brains are digital information directly from or to the robot, however the learning is done via biological means (the brain grows and the neurons develop over time, learning tasks that it repeats during infancy).

Currently my studies are in Artificial Intelligence in the form of Neural Networks and I will post more about these soon, but the premise of a neural network is that a collection of programmed "neuron" cells are used to learn and solve tasks. There are many ways to achieve this and I will only post about a few.

What can artificial intelligence be used for?
Systems that implement AI in its many forms have endless numbers of usages. For example Neural Networks are used in weather forecasting, handwriting recognition, voice pattern recognition and data analysis. For example CERN uses neural networks to categories collision events and only saves those which the network suggests are "interesting". This still produces terabytes of data however is far more manageable than if they where to store all collision data generated in the detectors.

Other types of AI can be used in production lines, ranging from basic repeated task code to control robotic construction arms to quality control testing (here if the output is 1 the item is good, otherwise it has failed). AI has even made it's way into day to day life, with services such as Siri of Google's voice search.

What are the big questions in the field of Artificial Intelligence?
Perhaps more interesting sometimes than the material itself is the "big" questions in the field of AI. These are very philosophical and ethical questions that researchers in the field must be aware of and sometimes try to solve. I will go through a few of them...

Are machines really intelligent?

This is a question that a lot of people ask, is there such a thing as Artificial Intelligence? The answer to this question not only depends on your definition of intelligence, but how you actually interpreted the question. For example, some would say anything that possesses the ability to learn is intelligent. As such neural networks are intelligent. Others do not agree! Alan Turing attempted to answer this question himself back in the early days of computer development. Turing noted that intelligence comes in different forms, and to program a computer to play chess at a high level was a trivial task... it does not show intelligence, simply computation. For Turing the test of intelligence was "Can machines think?" or rather as he later phrased it "can machines do what we do?" - As mental and physical activities are not entirely separate issues Turing developed what he referred to as the "imagination game" (You can look this up yourself.) This was later developed into what is now known as the Turing test and has been considered the true test of intelligence for many years. Turing predicted that computers would beat this test by the year 2000. The latest estimates put the date at 2029.

Due to the definition given by Alan Turing of what artificial intelligence should be, smaller levels of intelligence are generally ignored. Being able to predict the weather with a 99% accuracy using a neural network is not considered to be a great achievement, nor an act of an intelligent system, even though it is. The human centered approach to intelligence still very much drives forward research in artificial intelligence and the Turing test is still the standard test any "intelligent" system should pass.

Should human brain-cells be allowed to be used to create intelligent systems?

This is an issue that is very much under discussion at the moment. As humans believe our brain to be unique and as human nature is bias toward our species, unlike when other animal brain cells are used in experiments, human brain-cells cause a rather large discussion. Should we be allowed to grow brains from human brain-cells if they are simply going to be used to control a robot? This is a hard question to answer affirmatively without upsetting religious and ethical groups, for the sanctity of life is important and by growing a human brain, unless there is a sure knowledge that it will not be self aware, this usage of it could be considered to be cruel. On the other hand, as we do not know if the brain is sentient and we assume that it is not, by growing a human brain and using it to control systems, this gives an unprecedented opportunity to study the development and workings of the brain. Not only does this research further the advancement of artificially intelligent systems, but also the research can be used for medical purposes. If we know what causes issues with the brain, we may get a better understanding of how to solve them.

If a machine is considered intelligent should it be granted the same rights as a human?

This question is often ignored, as computers are seen very much as the "work horse" of the 21st century, however if an artificially intelligence system was developed, that could pass the Turing test, would that be enough to consider it as having equal status to humans? The answer to this question is hard to predict as the stage of intelligence here has yet to be achieved, however, once it is, what will happen to the system generated. The likely hood is, by looking through human history, an artificial intelligence will be a slave to humanity until the general public can be convinced that it is equal.

Should the military use artificial intelligence in drones?

This is a question I have very strong views on. The military are currently developing AI systems to pilot and control drones. These automated systems can perform a variety of tasks from providing visual surveillance of an area to firing rockets in more modern versions. To me this is not acceptable. Relating this to the former question on artificial intelligence rights, military use of AI will give systems a bad reputation, as any instances of "miss-fires" due to mistakes made by artificial systems will cast negative views in the public eye. However these systems should not be put in this position of control and responsibility in the first place, if people want to kill each other that is fine, but please leave our computers out of it, they should never be made to take a side.

This is the end of the philosophical debates, I shall not be commenting on them further - If I do they will have their own post and will have gained relevance that makes them deserve talking about! Keep an eye out for the next article on neural networks!

7Nov/120

AD – Active Directory, What is it ?

Posted by Dark#Basics

ActiveDirectory

What is it ?

Active Directory enables administrators to force policies and settings in a company network.  You can think of AD as a certain catalogue or a book that holds all the information regarding users, computers, resources, etc. but also settings for the company domain. It makes it possible to provide access and set permission based on that information stored in the catalogue by using different methods like for example a security group.

The most important role of Active Directory is providing the authentication information for users, computers and the resources that are part of the network.

Forests and Domains

When installing Active Directory the first time you need to think of Domains and Forests. A forest is a security boundary. Objects in separate forests are not able to interact with each other, unless the administrators of each separate forest create a trust between them. For example, an Enterprise Administrator account for domain1.com, which is normally the most privileged account of a forest, will have, no permissions at all in a second forest named domain2.com, even if those forests exist within the same LAN, unless there is a trust in place.

A domain is a management boundary. Domains are part of a forest. The first domain in a forest is known as the forest root domain. In many small and medium organizations (and even some large ones), you will only find a single domain in a single forest. The forest root domain defines the default namespace for the forest. For example, if the first domain in a new forest is named domain1.com, then that is the forest root domain. If you have a business need for a child domain, for example - a branch office in Chicago, you might name the child domain chi. The FQDN of the child domain would be chi.domain1.com. You can see that the child domain's name was prepended forest root domain's name. This is typically how it works. You can have disjoint namespaces in the same forest, but that's a whole separate can of worms for a different time.

In most cases, you'll want to try and do everything possible to have a single AD domain. It simplifies management, and modern versions of AD make it very easy to delegate control based on OU, which lessens the need for child domains.

Domain Names

I can name my domain whatever I want, right? Not really. dcpromo.exe, the tool that handles the promotion of a server to a DC isn't idiot-proof. It does let you make bad decisions with your naming, so pay attention to this section if you are unsure.

First of all, don't use made up TLDs like .local, .lan, .corp, or any of that other crap. Those TLDs are not reserved. ICANN is selling TLDs now, so your mycompany.corp that you're using today could actually belong to someone tomorrow. If you own mycompany.com, then the smart thing to do is use something like internal.mycompany.com or ad.mycompany.com for your internal AD name. If you use mycompany.com as an externally resolvable website, you should avoid using that as your internal AD name as well, since you'll end up with a split-brain DNS.

FSMO Roles

When a user tries to log in to a computer that is joined to AD using their AD credentials, the salted and hashed username and password combination are sent to the DC for both the user account and the computer account that are logging in. Yes, the computer logs in too. This is important, because if something happens to the computer account in AD, like someone resets the account or deletes it, you may get an error that say that a trust relationship doesn't exist between the computer and the domain. Even though your network credentials are fine, the computer is no longer trusted to log into the domain.

A server that responds to authentication or authorization requests is a Domain Controller (DC). In most cases, a Domain Controller will hold a copy of the Global Catalog. A Global Catalog (GC) is a partial set of objects in all domains in a forest. It is directly searchable, which means that cross-domain queries can usually be performed on a GC without needing a referral to a DC in the target domain. If a DC is queried on port 3268 (3269 if using SSL), then the GC is being queried. If port 389 (636 if using SSL) is queried, then a standard LDAP query is being used and objects existing in other domains may require a referral.

In general there are five functions that are needed to provide a fully functional Active Directory. The 5 roles and their function are:

  • Domain Naming Master - There is only one Domain Naming Master per forest. The Domain Naming Master makes sure that when a new domain is added to a forest that it is unique. If the server holding this role is offline, you won't be able to make changes to the AD namespace, which includes things like adding new child domains.
  • Schema Master - There is only one Schema Operations Master in a forest. It is responsible for updating the Active Directory Schema. Tasks that require this, such as preparing AD for a new version of Windows Server functioning as a DC or the installation of Exchange, require Schema modifications. These modifications must be done from the Schema Master.
  • Infrastructure Master - There is one Infrastructure Master per domain. If you only have a single domain in your forest, you don't really need to worry about it. If you have multiple forests, then you should make sure that this role is not held by a server that is also a GC holder. The infrastructure master is responsible for making sure that cross-domain references are handled properly. If a user in one domain is added to a group in another domain, the infrastructure master for the domains in question make sure that it is handled properly. This role will not function correctly if it is on a global catalog.
  • RID Master - The Relative ID Master (RID Master) is responsible for issuing RID pools to DCs. There is one RID master per domain. Any object in an AD domain has a unique Security Identifier (SID). This is made up of a combination of the domain identifier and a relative identifier. Every object in a given domain has the same domain identifier, so the relative identifier is what makes objects unique. Each DC has a pool of relative IDs to use, so when that DC creates a new object, it appends a RID that it hasn't used yet. Since DCs are issued non-overlapping pools, each RID should remain unique for the duration of the life of the domain. When a DC gets to ~100 RIDs left in its pool, it requests a new pool from the RID master. If the RID master is offline for an extended period of time, object creation may fail.
  • PDC Emulator - Finally, we get to the most widely misunderstood role of them all, the PDC Emulator role. There is one PDC Emulator per domain. If there is a failed authentication attempt, it is forwarded to the PDC Emulator. The PDC Emulator functions as the "tie-breaker" if a password was updated on one DC and hasn't yet replicated to the others. The PDC Emulator is also the server that controls time sync across the domain. All other DCs sync their time from the PDC Emulator. All clients sync their time from the DC that they logged in to. It's important that everything remain within 5 minutes of each other, otherwise Kerberos breaks and when that happens, everyone cries.

The important thing to remember is that the servers that these roles run on is not set in stone. It's usually trivial to move these roles around, so while some DCs do slightly more than others, if they go down for short periods of time, everything will usually function normally.

Primary and Secondary DC’s

The concept of PDCs and BDCs died with Windows NT4. If you create a secondary DC it will also be capable of offering authentication services. It’s best practice to have atleast two DCs per domain. These DCs should both have a copy of the GC and should both be DNS servers so that the network resources (Clients, Computers,..) are still able to query the DC.

The DCs belonging to the same domain in the same site will replicate their data to each other at a 15 second interval. Do note that there are urgent event that trigger replication after the data has been changed. Think things like password resets, account lockouts.

How can clients find the domain ?

Client and other resources can find the DCs by using DNS. It’s the most critical role that needs to function properly to have a functioning AD. Do note that while it is possible to use any type of DNS service it’s best to stick with using AD integrated DNS zones to avoid any DNS related problems.

When adding a client to the domain the first thing you always need to check is the DNS servers that client is using. Make sure it’s using the in house DNS servers because when trying to add the client to the domain it will try to resolve the domain name to locate the domain controller.

Each FSMO role will have a SRV DNS record that will point to client to the correct DC.

  • _ldap._tcp.<DNSDomainName> - Enables a client to locate a W2K domain controller in the domain named by <DNSDomainName>. A client searching for a domain controller in the domain dpetri.net would query the DNS server for _ldap._tcp.domain1.com.
  •  _ldap._tcp.<SiteName>._sites.<DNSDomainName> - Enables a client to find a W2K domain controller in the domain and site specified (e.g., _ldap._tcp.lab._sites. domain1.com for a domain controller in the Lab site).
  • _ldap._tcp.pdc._ms-dcs.<DNSDomainName> - Enables a client to find the PDC flexible single master object (FSMO) role holder of a mixed-mode domain. Only the PDC of the domain registers this record.
  •  _ldap._tcp.gc._msdcs.<DNSTreeName> - Enables a client to find a Global Catalog (GC) server. Only domain controllers serving as GC servers for the tree will register this name. If a server ceases to be a GC server, the server will deregister the record.
  •  _ldap._tcp. ._sites.gc._msdcs.<DNSTreeName> - Enables a client to find a GC server in the specified site (e.g., _ldap._tcp.lab._sites.gc._msdcs. domain1.com).
  •  _ldap._tcp.<DomainGuid>.domains._msdcs.<DNSTreeName> - Enables a client to find a domain controller in a domain based on the domain controller’s globally unique ID. A GUID is a 128-bit (8 byte) number that generates automatically for referencing Active Directory objects.
  • <DNSDomainName> - Enables a client to find a domain controller through a normal Host record.

Installation and backup of Active Directory.

More information regarding installation of Active Directory can be found here darkdevelopments.org/2011/08/30/win2008-install-active-directory/. Regarding the backup procedure of Active Directory more information can be found here darkdevelopments.org/2011/07/18/db-backuprecovery-notes-part1-active-directory-2/

[1] serverfault.com/questions/402580/what-is-active-directory-and-how-does-it-work
[2] www.petri.co.il/active_directory_srv_records.htm

7Nov/120

The Joy of Logical Volumes.

Posted by dijit

Linux logical volumes sound scary, but I can get you going in three simple commands, and the benefits are huge.

I was presenting at a conference recently where I happened to mention the Linux logical volume manager (“LVM”). I was surprised at people’s reactions. Most people that manage Linux have heard of it, but it seems that a lot of folks have never used it. Let me tell you, you’ve got to try this. It’s easy, it’s quick, and it makes life so much easier. You can create and delete partitions (really, “logical volumes”) on the fly, have ones that are larger than your largest physical disk, and grow and shrink them at will. It’s really good stuff.

Three Simple Terms

There are three terms you need to understand, and they’re not very complicated. Here they are:

  • Physical Volume: While this sounds like a physical disk, it’s really just disk storage. It can be a whole disk, but typically it’s just a partition or an MD RAID device.
  • Volume Group: This just combines the space of one or more physical volumes into a single pool of space. I usually just have one of these, but you can have more if you want to maintain separate storage pools (i.e. fast disks vs. slow disks, RAIDed vs. non-RAIDed, etc.).
  • Logical Volume: This is the functional equivalent to a partition, and we use it wherever we would normally use a partition. It’s a bunch of space allocated from a volume group.

Three Simple Commands

I promised three commands to go from nothing to done, so let’s do it.

  1. Create the physical volume: This is a brief step that labels the physical disk storage for use with the logical volume manager. In the simplest case, this is an existing but unused partition. Let’s assume that /dev/sda6 is my unused partition. To turn it into a LVM physical volume, just enter: 
    pvcreate /dev/sda6

     

    Repeat this command for each physical volume you want to create.

  2. Create the volume group: Now that we’ve created our physical volume(s), we need to create a volume group. This creates our pool of disk space. Let’s assume I’m going to call my volume group “workspace”. To create the “workspace” volume group using the physical volume I just created, I use this command: 
    vgcreate workspace /dev/sda6

     

    If I had multiple physical volumes, I would just list them all on the command line. I can also add and remove physical volumes on the fly as my needs change.

  3. Create the logical volume: Now that I have my volume group, I’m ready to create my first logical volume. The logical volume can be any size I want up to the amount of space available in the volume group, even if that space spans multiple physical volumes. Let’s assume that I want my logical volume to be called “userdata”, and I want it to be 30G in size. Here’s my command: 
    lvcreate -n userdata -L 30G workspace

     

    Use another lvcreate command any time you need a new “partition”.

That’s it. I now have a new logical volume called /dev/workspace/userdata that I can use just like a partition. I can format it with any file system I like, mount it, add it to /etc/fstab, use it for swap space, whatever.

So Why Is This Good?

At first glance, it seems we haven’t done much more than we could have done with ordinary partitions. What are the benefits of these extra couple of steps? Well, here are a few:

  1. We didn’t take the system down to create our “userdata” logical volume. We can create as many logical volumes as space allows without downtime. This alone has probably reduced the late evenings I’ve had to work by 90% - and that is a very good thing in my book.
  2. We can delete logical volumes any time we want. That means we can recover wasted space and reallocate it elsewhere without downtime.
  3. We can change the size of logical volumes on the fly. Is /tmp getting too small for our needs? Just grow it with the lvresize. Or shrink /home if you allocated too much space.IMPORTANT CAVEAT: LVM doesn’t know anything about the data inside the logical volume. It’s up to you manage the file system properly. To increase a file system, you can extend the logical volume, but then you still need to use the appropriate file system tool in order for the file system to recognize that its “partition” just got larger. Even more important, when reducing space you need to reduce the file system size first, then reduce the logical volume size. I always make the file system a little smaller than intended, change the volume size, and then grow the file system to fit the available space. This assures that I don’t chop off the end of my file system.

    Reiser can shrink its file systems if they’re unmounted, but most file systems don’t have tools to reduce a file system in-place. In those cases you typically have to create a new, smaller logical volume, copy all the files over, and change the mount points. As always, take backups before doing anything risky, and shrinking a file system is definitely risky.

    None of this is different than it would be if you were managing real partitions. It’s just easier and quicker, and you don’t need to take the system down.

  4. As mentioned before, logical volumes can span physical volumes. You can get really big file systems this way, or put all those old, too-small disks back to work. Need a 10 terabyte file system? Done.
  5. We don’t have to worry about partition locations. With conventional partitions, I might have a partition /dev/sdb5 that is too small, and have available space at the end of the disk that I’d like to allocate to it, but I have /dev/sdb6 in between. That means I have to take the system down, move /dev/sdb6 down to the end of the disk, and then grow /dev/sdb5. With the logical volume manager, available space is available space. You don’t need to worry about where it is.
  6. You can move logical volumes between physical disks. If you suspect a disk is beginning to fail, you can create a new physical volume on a different disk, add it to the volume group, and migrate all the logical volumes off of the old disk while they’re still in use!
  7. Is your backup window too small? The logical volume manager supports snapshots. A snapshot gives you a static copy of your logical volume to backup. Just quiesce your applications if necessary, take your snapshot, and resume normal operations. You can backup the static snapshot at your leisure and delete it when you’re done. Your backup outage just went from six hours to 90 seconds.

Summary

The Linux Logical Volume Manager is easy to set up, easy to use, reduces downtime, and reduces off-hours work. If you’ve got a partition available, I encourage you to try it right now. It’ll take you 10 minutes, and you’ll never go back.

 

[1] http://www.pdxsys.com/articles/lvm/

4Nov/121

Securing Fedora workstations for the enterprise.

Posted by dijit

Hi Guys,Today I'd like to talk about securing Fedora based GNU+Linux distro's for use with real people in real companies.

Generally I work on servers, however recently I've been tasked with managing workstations, and luckily I can dictate the operating system used (as long as it's functional)

I opt for fedora because it's stable while being bleeding edge, free (as in beer), and also supporting many of the features I've come to love of RHEL.

One of those features is "Kickstart" and when deployed with a central repository, you can have very powerful and scalable build system where by you can do hotdesking and can each "node" can be destroyed and rebuilt with non-moving targets meaning you know the current state of a new build.

This process is very fast, and automated (<3).

I won't go into detail explaining why that's a good thing, a sysadmin brain would know, so I wont bore you... instead I'm going to try to explain my kickstart file...

Select All Code:
install
lang en_GB.UTF-8
keyboard uk
network --onboot yes --device eth0 --bootproto dhcp --noipv6 --hostname testbox.virt.drk.sc

Standard stuff, only things that might need to change are, hostname, and of course localisation

next we configure the timezone and the root user:

Select All Code:
timezone --utc Europe/London
rootpw  --iscrypted $(mkpasswd -m sha-512 $password $salt)
selinux --enforcing
authconfig --enableshadow --passalgo=sha512
firewall --service=ssh

we're also allowing SSH through the local firewall because we want to remotely control these machines.

Next we need to configure some disks, since /dev/sda is used for most drives, and these machines will be commodity based, we'll assume you have a /dev/sda

Select All Code:
12
13
14
15
16
17
zerombr
clearpart --all --drives=sda
part /home --fstype=ext4 --grow --size=500
part / --fstype=ext4 --grow --size=1024
part /boot --fstype=ext2 --size=500 --asprimary
bootloader --location=mbr --timeout=5 --driveorder=sda,sdb,sdc --append="rhgb quiet"

next we configure our local repos, here we're assuming that the name of your server is "localrepo" and that it's using mDNS

Select All Code:
20
21
repo --name="Fedora 17 - x86_64"  --baseurl=http://localrepo.local/fedora/linux/releases/17/Everything/x86_64/os/ --cost=1000
repo --name="Fedora 17 - x86_64 - Updates"  --baseurl=http://localrepo.local/fedora/linux/updates/17/x86_64/ --cost=1000

now we're going to install our packages and set our security options in the %pre and %post sections:

Select All Code:
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
%packages
@admin-tools
@base
@core
@editors
@system-tools
@text-internet
ntop
dbench
xfsprogs
mtools
syslinux
iscsi-initiator-utils
ksh
squashfs-tools
hardlink
x86info
jfsutils
htop
geoclue
lzop
p7zip-plugins
pidgin
iftop
iotop
syslog-ng
ipsec-tools
p7zip
thunderbird
rootsh
gnutls-utils
fuse
fdupes
irssi
iperf
ncftp
vim
-fprintd-pam # I've not seen anyone use fingerprinters
-evolution # thunderbird is superior
-empathy   # pidgin is superior
-transmission-gtk
-evolution-NetworkManager
-evolution-help
-cadaver
-xorg-x11-drv-wacom
-xorg-x11-drv-vmware
-xorg-x11-drv-openchrome
-xorg-x11-drv-qxl
-xorg-x11-drv-vmmouse
-xorg-x11-drv-nouveau
 
# Removing some packages from those groups above.
-xinetd
-telnet-server
-telnet
-krb5-workstation
-rsh-server
-rsh
-tftp-server
# CCE-14495-6 (row 222)
-sendmail
# CCE-4464-4 (row 219)
-dhcp
# CCE-14881-7 (row 240)
-vsftpd
# CCE-4514-6 (row 241)
-httpd
-gnome-user-share
# CCE-14825-4 (row 178)
-isdn4k-utils
# CCE-17504-2 (row 255)
-irda-utils
# CCE-18200-6 (row 253)
-talk
## If you have a 32 bit system, comment out the next line
-*.i?86
# CCE-18031-5 (row 250)
-ipsec-tools
# CCE-17250-2 (row 251)
-pam_ccreds
# FIXME: need row
openswan
# CCE-17742-8 (row 134)
-sysklogd
rsyslog
 
# Post-install commands
 
# Some post-installation configuration can be done from the kickstart file
# itself.  These actions should not be relied upon for system 
# configuration/management.  Anything in the %post section should be things
# that would immediately be done after installation that are either out of
# scope for the management software, or help prepare the system for the
# management software.
 
%post
# Install redhat-release key for later use validating rpms
# CCE-14440-2 (row 7)
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-x86_64
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora
 
# Disable yum-updatesd daemon (CCE-4218-4  row 10)
chkconfig yum-updatesd off
 
# Notes CCE-14914-6, CCE-14813-0, CCE-14931-0 (row 11, 12, and 14 are noops)
 
# Fix up the partitions to be secure
# CCE    (rows 15 - 25)
FSTAB=/etc/fstab
# nodev, noexec, and nosuid on /boot
TEST="`grep ' \/boot ' ${FSTAB} | grep -c 'noexec'`"
if [ "$TEST" = "0" ]; then
        MNT_OPTS=$(grep " \/boot " ${FSTAB} | awk '{print $4}')
        sed -i "s/\( \/boot.*${MNT_OPTS}\)/\1,nodev,noexec,nosuid/" ${FSTAB}
fi
# nodev, noexec, and nosuid on /dev/shm
# CCE-15007-8, CCE-14306-5, CCE-14703-3 (Rows 22 - 24)
TEST="`grep ' \/dev\/shm ' ${FSTAB} | grep -c 'noexec'`"
if [ "$TEST" = "0" ]; then
        MNT_OPTS=$(grep " \/dev\/shm " ${FSTAB} | awk '{print $4}')
        sed -i "s/\( \/dev\/shm.*${MNT_OPTS}\)/\1,nodev,noexec,nosuid/" ${FSTAB}
fi
# Make /var/tmp use /tmp
# CCE-14584-7 (Row 25)
grep " \/var\/tmp " ${FSTAB} >/dev/null
if [ $? -eq 1 ]; then
        echo -e "/tmp\t\t/var/tmp\t\t\text3\tdefaults,bind,nodev,noexec,nosuid\t0 0" >> ${FSTAB}
fi
 
# Don't use modprobe.conf, put changes in 1 place
touch /etc/modprobe.d/usgcb-blacklist
 
# Disable mounting of cramfs CCE-14089-7 (row 26)
echo -e "install cramfs /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of freevxfs CCE-14457-6 (row 27)
echo -e "install freevxfs /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of hfs CCE-15087-0 (row 28)
echo -e "install hfs /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of hfsplus CCE-14093-9 (row 29)
echo -e "install hfsplus /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of jffs2 CCE-14853-6 (row 30)
echo -e "install jffs2 /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of squashfs CCE-14118-4 (row 31)
echo -e "install squashfs /bin/true" >> /etc/modprobe.d/usgcb-blacklist
# Disable mounting of udf CCE-14871-8 (row 32)
echo -e "install udf /bin/true" >> /etc/modprobe.d/usgcb-blacklist
 
# Notes (row 33 - 51 are noops)
 
# CCE-4220-0  (Row 52)
echo -e "umask 027" >> /etc/sysconfig/init
 
# CCE-4225-9 (Row 53)
echo -n "* hard core 0" >> /etc/security/limits.conf
 
# Notes CCE-4225-9, CCE-4146-7, CCE-4172-3 (row 54 -57 are noops)
 
# CCE-3485-0, CCE-4256-4  (Rows 58 & 59)
sed -i "/^vc/d" /etc/securetty
 
# CCE-15047-4 (Row 60)
sed -i "6s/^#//" /etc/pam.d/su
 
# Notes CCE-14088-9 (row 61 is noop)
 
# Notes CCE-3987-5, CCE-4238-2, CCE-14300-8, CCE-4009-7 (rows 62 - 65 are noops)
 
# CCE-4180-6 (Row 66)
sed -i "/PASS_MIN_DAYS/s/[0-9]/1/" /etc/login.defs
 
# CCE-4097-2 (Row 67)
sed -i "/PASS_WARN_AGE/s/[0-9]/14/" /etc/login.defs
 
# CCE-4092-3 (Row 68)
sed -i "/PASS_MAX_DAYS/s/[0-9]\{5\}/60/" /etc/login.defs
 
# CCE-4154-1 (Row 69)
sed -i "/PASS_MIN_LEN/s/[0-9]/12/" /etc/login.defs
 
# Notes CCE-14675-3, CCE-4114-5, CCE-14071-5 (rows 70 - 72 are noops)
 
# The following line covers
# (rows 73 - 78)
sed -i "/pam_cracklib.so/s/retry=3/retry=3 minlen=12 dcredit=-1 ucredit=-1 ocredit=-1 lcredit=-1 difok=3/" /etc/pam.d/system-auth
 
# CCE-3410-8 (row 79) system-auth
sed -i "5i\auth\trequired\tpam_tally2.so deny=5 onerr=fail" /etc/pam.d/system-auth
#sed -i "/^auth/s/sufficient/required/" /etc/pam.d/system-auth
##sed -i "/^auth/s/requisite/required/" /etc/pam.d/system-auth
#sed -i "/^auth/d/requisite/" /etc/pam.d/system-auth
#sed -i "/pam_deny/d" /etc/pam.d/system-auth
 
# The old way
#sed -i "/^auth/s/include/required\tpam_tally2.so deny=5 onerr=fail\nauth\tinclude\t/" /etc/pam.d/gdm
#sed -i "/^auth/s/include/required\tpam_tally2.so deny=5 onerr=fail\nauth\tinclude\t/" /etc/pam.d/sshd
#sed -i "/^auth/s/include/required\tpam_tally2.so deny=5 onerr=fail\nauth\tinclude\t/" /etc/pam.d/login
 
# CCE-14063-2(row 80) is a noop since this is the defaults
 
# CCE-14939-3 (row 81)
sed -i "/pam_unix.so/s/shadow/shadow remember=24/" /etc/pam.d/system-auth
 
# Notes CCE-3301-9, CCE-14957-5, CCE-4090-7 (rows 82 - 84 are noops)
 
# CCE-14107-7 (row 85)
sed -i "/UMASK/s/[0-9]\{3\}/077/" /etc/login.defs
 
# CCE-14847-8 (row 86)
echo "umask 077" >> /etc/profile
 
# CCE-3844-8  (row 87)
sed -i "/umask/s/022/077/" /etc/bashrc
 
# CCE-4227-5  (row 88)
sed -i "/umask/s/022/077/" /etc/csh.cshrc
 
# Notes CCE-3923-0  (rows 89 is a noop)
 
# Notes CCE-4197-0, CCE-4144-2  (rows 91 - 92 are noops)
 
# CCE-4241-6 (row 93)
echo "~:S:wait:/sbin/sulogin" >> /etc/inittab
 
# CCE-4245-7 (row 94)
sed -i "/PROMPT/s/yes/no/" /etc/sysconfig/init
 
# CCE-3315-9 (row 95)
gconftool-2 --direct \
              --config-source xml:readwrite:/etc/gconf/gconf.xml.mandatory \
              --type int \
              --set /apps/gnome-screensaver/idle_delay 15
 
# CCE-14604-3 (row 96)
gconftool-2 --direct \
              --config-source xml:readwrite:/etc/gconf/gconf.xml.mandatory \
              --type bool \
              --set /apps/gnome-screensaver/idle_activation_enabled true
 
# CCE-14023-6 (row 97)
gconftool-2 --direct \
              --config-source xml:readwrite:/etc/gconf/gconf.xml.mandatory \
              --type bool \
              --set /apps/gnome-screensaver/lock_enabled true
 
# CCE-14735-5 (row 98)
gconftool-2 --direct \
              --config-source xml:readwrite:/etc/gconf/gconf.xml.mandatory \
              --type string \
              --set /apps/gnome-screensaver/mode blank-only
 
# CCE-4060-0 (row 100)
echo -e "\n-- WARNING --\nThis system is for the use of authorized users only. Individuals\nusing this computer system without authority or in excess of their\nauthority are subject to having all their activities on this system\nmonitored and recorded by system personnel. Anyone using this\nsystem expressly consents to such monitoring and is advised that\nif such monitoring reveals possible evidence of criminal activity\nsystem personal may provide the evidence of such monitoring to law\nenforcement officials.\n" > /etc/issue
 
# CCE-4188-9 (row 101)
sed -i "15s//\n        \n        \n            \n            \n            \n-- WARNING --\nThis system is for the use of authorized users only. Individuals\nusing this computer system without authority or in excess of their\nauthority are subject to having all their activities on this system\nmonitored and recorded by system personnel. Anyone using this\nsystem expressly consents to such monitoring and is advised that\nif such monitoring reveals possible evidence of criminal activity\nsystem personal may provide the evidence of such monitoring to law\nenforcement officials.\n            <\/text>\n            <\/item>\n        <\/box>\n    <\/item>\n\n    /" /usr/share/gdm/themes/Fedora/Fedora.xml
 
# CCE-3977-6, CCE-3999-0, and CCE-3624-4 (rows 102 - 104) are noops
 
# CCE-3668-1 (row 105)
chkconfig mcstrans off
 
# CCE-14991-4 (row 106) is noop
 
# CCE-3561-8 (row 107)
echo -e "\n# Changes for USGCB content" >> /etc/sysctl.conf
echo "net.ipv4.ip_forward = 0" >> /etc/sysctl.conf
 
# CCE-4155-8 (row 108)
echo "net.ipv4.conf.all.send_redirects = 0" >> /etc/sysctl.conf
 
# CCE-4151-7 (row 109)
echo "net.ipv4.conf.default.send_redirects = 0" >> /etc/sysctl.conf
 
# CCE-3472-8 (row 110)
echo "net.ipv4.conf.all.secure_redirects = 0" >> /etc/sysctl.conf
 
# CCE-4217-6 (row 111)
echo "net.ipv4.conf.all.accept_redirects = 0" >> /etc/sysctl.conf
 
# CCE-4236-6 (row 112)
echo "net.ipv4.conf.all.accept_source_route = 0" >> /etc/sysctl.conf
 
# CCE-3339-9 (row 113)
echo "net.ipv4.conf.default.secure_redirects = 0" >> /etc/sysctl.conf
 
# CCE-4186-3 (row 114)
echo "net.ipv4.conf.default.accept_redirects = 0" >> /etc/sysctl.conf
 
# CCE-4091-5 (row 115)
echo "net.ipv4.conf.default.accept_source_route = 0" >> /etc/sysctl.conf
 
# CCE-4133-5 (row 116)
echo "net.ipv4.icmp_ignore_bogus_error_responses = 1" >> /etc/sysctl.conf
 
# CCE-3644-2 (row 117)
echo "net.ipv4.icmp_echo_ignore_broadcasts = 1" >> /etc/sysctl.conf
 
# CCE-4320-8 (row 118)
echo "net.ipv4.conf.all.log_martians = 1" >> /etc/sysctl.conf
 
# CCE-4080-8 (row 119)
echo "net.ipv4.conf.all.rp_filter = 1" >> /etc/sysctl.conf
 
# CCE-4265-5 (row 120)
echo "net.ipv4.tcp_syncookies = 1" >> /etc/sysctl.conf
 
# CCE-3840-6 (row 121)
echo "net.ipv4.conf.default.rp_filter = 1" >> /etc/sysctl.conf
 
# CCE-15013-6, CCE-4276-2 (rows 122 and 123) are noops
 
# CCE-18455-6 (row 124)
echo -e "options ipv6 disable=1" >> /etc/modprobe.d/usgcb-blacklist
 
# CCE-4313-3 (row 125)
echo "net.ipv6.conf.default.accept_redirect=0" >> /etc/sysctl.conf
 
# CCE-4269-7 (row 126)
echo "net.ipv6.conf.default.accept_ra=0" >> /etc/sysctl.conf
 
# CCE-4167-3 (row 127)
# This is being set to off because IPv6 is disabled
chkconfig ip6tables off
 
# CCE-4189-7 (row 128)
chkconfig iptables on
 
# CCE-14264-6 (row 129)
sed -i "/^:INPUT/s/ACCEPT/DROP/" /etc/sysconfig/iptables
 
# CCE-14268-7 (row 130)
echo -e "install dccp /bin/true" >> /etc/modprobe.d/usgcb-blacklist
 
# CCE-14235-5 (row 131)
echo -e "install sctp /bin/true" >> /etc/modprobe.d/usgcb-blacklist
 
#i CCE-14027-7 (row 132)
echo -e "install rds /bin/true" >> /etc/modprobe.d/usgcb-blacklist
 
# CCE-14911-2 (row 133)
echo -e "install tipc /bin/true" >> /etc/modprobe.d/usgcb-blacklist
 
# CCE-17698-2 (row 135)
chkconfig rsyslog on
chkconfig rsyslog --levels 345 on
 
# (rows 136 - 138) are noops
 
# send logging to remote server CCE-17248-6 (row 139)
mkdir -m 0700 /etc/pki/rsyslog
##
## The following lines need site specific customizations
##
#echo "" >> /etc/rsyslog.conf
#echo '# make gtls driver the default' >> /etc/rsyslog.conf
#echo '$DefaultNetstreamDriver gtls' >> /etc/rsyslog.conf
#echo "" >> /etc/rsyslog.conf
#echo '# certificate files' >> /etc/rsyslog.conf
#echo '$DefaultNetstreamDriverCAFile /etc/pki/rsyslog/ca.pem' >> /etc/rsyslog.conf
#echo '$DefaultNetstreamDriverCertFile /etc/pki/rsyslog/machine-cert.pem' >> /etc/rsyslog.conf
#echo '$DefaultNetstreamDriverKeyFile /etc/pki/rsyslog/machine-key.pem' >> /etc/rsyslog.conf
#echo "" >> /etc/rsyslog.conf
#echo '$ActionSendStreamDriverAuthMode x509/name' >> /etc/rsyslog.conf
#echo '$ActionSendStreamDriverPermittedPeer central.example.net' >> /etc/rsyslog.conf
#echo '$ActionSendStreamDriverMode 1 # run driver in TLS-only mode' >> /etc/rsyslog.conf
#echo '*.* @@central.example.net:10000 # forward everything to remote server port 10000' >> /etc/rsyslog.conf
 
# CCE-17639-6 (row 140) is a noop
 
# CCE-4182-2 (row 141) is noop
 
# CCE-4292-9 (row 142)
chkconfig auditd on
 
# (rows 144 - 151, 153 - 155) 
FILE=`rpm -ql audit | grep stig`
if [ x"$FILE" != "x" ] ; then
	cat $FILE | egrep -v 'immutable|ping|-e 2' > /etc/audit/audit.rules
fi
 
sed -i -e 's/^#\(-a always,exit -F arch=b.. -S clock_settime\)/\1 -F a0=0/g' /etc/audit/audit.rules
 
# CCE-14296-8 (row 152)
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f | awk '{printf "-a always,exit -F path=%s -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged\n", $1 }' >> /etc/audit/audit.rules
echo -e "\n" >> /etc/audit/audit.rules
 
# CCE-14688-6 (row 156)
echo -e "-w /sbin/insmod -p x -k modules" >> /etc/audit/audit.rules
echo -e "-w /sbin/rmmod -p x -k modules" >> /etc/audit/audit.rules
echo -e "-w /sbin/modprobe -p x -k modules" >> /etc/audit/audit.rules
echo -e "-a always,exit -F arch=b32 -S init_module -S delete_module -k modules" >> /etc/audit/audit.rules
echo -e "-a always,exit -F arch=b64 -S init_module -S delete_module -k modules" >> /etc/audit/audit.rules
echo -e "\n" >> /etc/audit/audit.rules
 
# CCE-14692-8 (row 157)
echo -e "-e 2" >> /etc/audit/audit.rules
 
# (rows 158 - 171) are noops
 
# CCE-4421-4 (row 172)
chkconfig readahead_early off
 
# CCE-4302-6 (row 173)
chkconfig readahead_later off
 
# CCE-4355-4 (row 174)
chkconfig bluetooth off
 
# CCE-4377-8 (row 175)
chkconfig hidd off
 
# CCE-14948-4 (row 176)
echo "alias net-pf-31 off" >> /etc/modprobe.d/usgcb-blacklist
echo "alias bluetooth off" >> /etc/modprobe.d/usgcb-blacklist
 
# CCE-4286-1, CCE-14825-4, CCE-3425-6 (row 177 - 179) are noops
 
# CCE-14054-1 (row 180)
echo "NOZEROCONF=yes" >> /etc/sysconfig/network
 
# CCE-4324-0 row 181 is a noop
 
# CCE-4304-2 (row 182)
chmod 0600 /etc/anacrontab
 
# CCE-4388-5 (row 183)
chmod 0600 /etc/crontab
 
# CCE-4250-7 (row 184) is a noop
 
# CCE-4450-3 (row 185)
chmod 0700 /etc/cron.daily
 
# CCE-4106-1 (row 186)
chmod 0700 /etc/cron.hourly
 
# CCE-4251-5 (row 187)
chmod 0700 /etc/cron.monthly
 
# CCE-4203-6 (row 188)
chmod 0700 /etc/cron.weekly
 
# (rows 189 - 202) are noops
 
# CCE-14466-7 (row 203)
chkconfig atd off
 
# CCE-4325-7 (row 204) is a noop
 
# CCE-14061-6 (row 205)
sed -i "s/#ClientAliveCountMax 3/ClientAliveCountMax 0/" /etc/ssh/sshd_config
 
# CCE-3845-5 (row 206)
sed -i "s/#ClientAliveInterval 0/ClientAliveInterval 900/" /etc/ssh/sshd_config
 
# CCE-4475-0, CCE-4370-3 (rows 207- 208) are noop
 
# CCE-4387-7 (row 209)
sed -i "s/#PermitRootLogin yes/PermitRootLogin no/" /etc/ssh/sshd_config
 
# CCE-3660-8 (row 210) is a noop
 
# CCE-4431-8 (row 211)
sed -i "s/#Banner \/some\/path/Banner \/etc\/issue/" /etc/ssh/sshd_config
 
# CCE-14716-5 (row 212) is noop
 
# CCE-14491-5 (row 213)
echo "Ciphers aes128-ctr,aes192-ctr,aes256-ctr" >> /etc/ssh/sshd_config
 
# CCE-4074-1 (row 214)
echo "exec X :0 -nolisten tcp \$@" > /etc/X11/xinit/xserverrc
 
# CCE-3717-6 (row 215)
sed -i "s/\[greeter\]/\[greeter\]\nInfoMsgFile=\/etc\/issue\n/" /etc/gdm/custom.conf
 
# CCE-4365-3 (row 216)
chkconfig avahi-daemon off
 
# CCE-4425-5 (row 217)
chkconfig hplip off
 
# CCE-4336-4 (row 218) noop due to (row 219)
 
# CCE-4376-0 (row 220)
chkconfig ntpd on
 
# CCE-4385-1 (row 221) ntp.conf has some ntp servers in it
 
# CCE-15018-5 (row 224) is a noop
 
# CCE-14894-0 (row 225) 
sed -i "s/#ssl start_tls/ssl start_tls/" /etc/ldap.conf
sed -i "s/#tls_checkpeer/tls_checkpeer/" /etc/ldap.conf
sed -i "s/#tls_cacertdir \/etc\/ssl\/certs/tls_cacertdir \/etc\/pki\/tls\/CA/" /etc/ldap.conf
#sed -i "s/#tls_cacertfile \/etc\/ssl\/ca.cert\/tls_cacertfile \/etc\/pki\/tls\/CA\/cacert.pem/" /etc/ldap.conf
sed -i "s/#tls_cacertfile \/etc\/ssl\/ca.cert/tls_cacertfile \/etc\/pki\/tls\/CA\/cacert.pem/" /etc/ldap.conf
 
# CCE-3501-4 (row 226) noop since openldap not installed
 
# CCE-4396-8 (row 227)
chkconfig nfslock off
 
# CCE-3535-2 (row 228)
chkconfig rpcgssd off
 
# CCE-3568-3 (row 229)
chkconfig rpcidmapd off
 
# CCE-4533-6 (row 230)
chkconfig netfs off
 
# CCE-4550-0 (row 231)
chkconfig portmap off
 
# CCE-4473-5 (row 232)
chkconfig nfs off
 
# CCE-4491-7 (row 233)
chkconfig rpcsvcgssd off
 
# CCE-4368-7, CCE-4024-6, CCE-3578-2 (rows 234 - 236) are noops
 
# CCE-3578-2 (row 237 & 238) noop
 
# CCE-3919-8 (row 239) noop since 243 has it uninstalled
 
# CCE-4338-0 (rows 240) is a noop since httpd not installed
 
# CCE-3847-1, CCE-4239-0 (rows 242 - 243) are noops since dovecot is not installed
 
# CCE-4551-8 (rows 244) is a noop since the server is not installed
 
# CCE-14075-6 (row 245)
sed -i "s/\[global\]/\[global\]\nclient signing = mandatory/" /etc/samba/smb.conf
 
# CCE-15029-1 (row 246) is a noop due to needing to be done in fstab
 
# CCE-4556-7, CCE-4076-6 (rows 247, 248) noops due to squid not being installed
 
# CCE-3765-5, CCE-14081-4 (rows 249, 250) noops since net-snmp is not installed
 
# CCE-18200-6 (row 252) is noop since talk-server is not installed
 
# CCE-17504-2 (row 253) is noop since irda-utils is not installed
 
# We turn this off since we already configured things
chkconfig firstboot off
 
# turn off selinux troubleshooter since root is needed
chkconfig setroubleshoot off
 
# CCE-3649-1 (row 254)
sed -i "/631/d"  /etc/sysconfig/iptables
 
# CCE-18037-2 (row 255)
sed -i "/5353/d"  /etc/sysconfig/iptables
 
# CCE-4072-5 (row 256)
chkconfig autofs off
 
# CCE-17816-0 (row 257)
chkconfig rawdevices off
 
# CCE-18412-7 (row 259)
useradd -D -f 30
 
# CCE-XXXXX-X (row XXX) disable gnome thumbnailers. Skipped for now.
#gconftool-2 --direct --config-source xml:readwrite:/etc/gconf/gconf.xml.mandatory --type bool --set /desktop/gnome/thumbnailers/disable_all true
 
# Workaround esound creating the directory in conflict with CCE-14794-2
mkdir -m 1777 /tmp/.esd

Please note: I have not configured LDAP here. that will come.

Filed under: Linux, Security 1 Comment